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ABSTRACT 

The  Navy's  various  weapons  project  officers 
routinely  decide  whether  to  use  existing  weapons 
technology  or  to  extend  into  as  yet  undeveloped 
technology.  For  state-of-the-art  ( SOA )  extensions, 
initial  estimates  of  development  cost  frequently  are 
inaccurate.  This  study  first  examines  the  background 
of  methods  utilized  for  SOA  extension  measurement. 
This  study  also  reviews  the  cost  estimating  methods 
used  by  Litton  Applied  Technology,  Inc.  to  estimate  the 
development  costs  of  the  AN/ALR-67  Radar  Warning 
Receiver,  a  specific  SOA  extension  project.  The 
principal  findings  are  that  regression  analysis  and 
geometric  surface  analysis  are  used  to  quantify  SOA 
extensions,  but  only  in  theoretical  applications. 
Litton  Applied  Technology  uses  the  bottoms-up  approach 
to  estimate  development  costs.  The  future  trends  in 
defense  cost  estimating  are  also  forecasted. 
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I .   INTRODUCTION 

A.  PURPOSE 

This  thesis  will  describe  how  Litton  Applied 
Technology  of  San  Jose,  California  measured  and 
estimated  the  cost  of  the  state-of-the-art  development 
of  the  AN/ALR-67  Radar  Warning  System.   This  case  study 
will  be  utilized  by  Professor  Willis  Greer  to  test  an 
improved  cost  estimating  model  which  will  relate 
measurements  of  state-of-the-art  extensions  with  costs. 

B.  BACKGROUND 

During  the  past  three  years,  budget  restraints  have 
forced  the  Navy  to  tighten  controls  over  expenditures 
for  major  weapons  systems.  One  area  which  has 
generated  significant  unplanned  cost  growth  during  the 
last  two  decades  has  been  the  extension  of  states-of- 
the-art  ( SOA )  .  The  cost  impact  of  SOA  extensions  is 
most  keenly  felt  during  the  demonstration  and 
validation  phase,  and  the  full  scale  development  phase 
of  the  acquisition  process.  During  an  interview  with 
GTE  cost  analyst  Stan  Swales  on  17  September  1987,  he 
estimated  most  cost  models  for  initial  SOA  development 
efforts  were  only  accurate  to  within  twenty  to  forty 
percent.  However,  once  the  generic  cost  models  were 
"calibrated"  with  actual  cost  parameters  and  the 
technical  characteristics  of  the  completed  system,  the 
range  of  accuracy  could  be  reduced  to  five  to  ten 
percent.   [Ref.  1] 

During  the  concept  exploration  process  of  new 
weapons  systems,  the  Navy  must  decide  if  the  risks  of 
unplanned  cost  growth  should  deter  extending  as-yet 
undeveloped  technology.  Prior  contractor  research, 
primarily    by     the    Rand    and    General    Research 
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Corporations,  has  made  progress  in  defining  and 
measuring  the  extent  of  technological  change  in  complex 
defense  systems,  primarily  aircraft  related.  These 
prior  studies  have  concentrated  on  the  generation  of 
cost  models  for  specific  applications  under  Department 
of  Defense  (DOD)  contracts,  rather  than  a  defined  cost 
model  for  general  use  by  Navy  systems  commands.  For 
example,  Dr.  E.N.  Dodson  developed  the  High-Energy 
Laser  Systems  Cost  Model  (Caliper  II)  for  the  Air  Force 
Weapons  Laboratory  at  Kirkland  Air  Force  Base,  New 
Mexico.  [Ref.  2]  However,  the  Navy  has  yet  to 
coordinate  new  cost  estimating  models  with  the  Air 
Force  . 

Cost  models  which  could  accurately  delineate  the 
association  between  levels  of  SOA  extensions  and  cost 
overruns  would  enhance  budgeting  accuracy.  Most  budget 
analysts  for  multimillion  dollar  weapons  systems 
utilize  the  Work  Breakdown  Structure  (WBS)  authorized 
by  the  DOD  Cost /Schedule  Control  Systems  Criteria. 
[Ref.  3]  Effective,  comprehensive  cost  models  are 
required  for  initial  hardware  development  cost 
estimates,  particularly  in  the  constantly  changing 
electronics  environment.  The  Navy's  negotiating 
capabilities  during  acquisition  for  new  technological 
systems  will  be  strengthened  with  increased  expertise 
in  SOA  extension  resource  estimation. 

One  field  rapidly  gaining  in  importance  and 
magnitude  is  software  cost  estimating,  which  experts 
like  Elmer  Branyan  from  General  Electric  predict  will 
account  for  eighty-five  percent  of  all  embedded 
computer  costs  by  1990.  [Ref.  4]  Defense  cost  models 
must  assimilate  and  utilize  the  latest  techniques 
incorporated   in    software   models   developed   by   RCA 


(Price-S  model)  and  Hughes  Aircraft  (developed  by  Dr. 
Jensen ) . 

Hardware  cost  estimation  for  initial  developed 
relies  on  three  primary  tools:  estimation  by  analogy, 
use  of  cost  estimating  relationships  (parametric 
costing),  and  systems  engineered  analysis  (detailed 
bottoms  up  estimation).  These  three  methods  are 
applied  on  the  basis  of  available  data,  which  will  be 
differentiated  and  described  in  Chapter  Two. 

The  following  quotation  by  Larry  Smith  emphasizes 
the  fundamental  nature  of  accurate  estimating: 

The  process  of  estimating  is  the  process  of   making  a 
prediction  or   forecast  of   predefined  events  and/or 
occurrences  weighted  and  influenced  by  subjective 
and  objective   information.    The  planning  techniques 
that  are  available  rely   on  the   estimates  to  develop 
schedules,  resource   histograms,  budgets,  cash  flow 
histograms,  and   performance  standards.    The  results 
and   sophisticated   methodology   are   only  as  good  as 
the  input  estimates.   The  time   spent  developing  good 
estimates  and   the  understanding   required  to  produce 
good  estimates,   will   reduce   the   need   for   major 
revisions  to  the  plan  and  schedule.   Good  estimates 
will   reduce   schedule   slippage   and  cost  overruns 
and  set  the  stage  to  facilitate  the  implementation  of 
project  plans.   [Ref.  5] 

C.   OBJECTIVE 

The  objective  of  this  thesis  is  to  describe  and 
analyze  how  defense  contractors  control  the  cost  of 
states-of-the-art  extensions.  Litton  Applied 
Technology,  located  in  San  Jose,  California  will  be  the 
subject  of  the  case  study.  This  thesis  compares 
Litton's  development  cost  estimate  of  the  AN/ALR-67 
Radar  Warning  System  with  their  actual  development 
costs.  Also,  Litton's  methodology  for  quantifying  the 
SOA  extension  to  the  AN/ALR-67  Radar  Warning  System  is 
analyzed. 

The  following  factors  integral  to  Litton's 
management  of  the  AN/ALR-67  Radar  Warning  System  are 
also  discussed: 
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1.  Litton's  cost  estimation  process  for  each  of  the 
following  acquisition  phases:  concept  exploration 
demonstration  and   validation,  full-scale 
development,  and  initial  production.    The 
process  description   will  indicate  which  cost 
techniques  were  utilized  by  Litton  for  each 
phase . 

2.  Litton's  use  of  modern  parametric  models  in 
hardware  and  software  cost  estimation. 

3.  Soundness  and  range  of  Litton's  control  system  in 
monitoring  and  reducing  cost,  schedule,  and 
performance  variances. 

4.  Specific  variance  analysis  by  phase, 
concentrating  on   the  effectiveness  of  Litton's 
cost  model . 

5.  External  factors  impacting  Litton's  current 
initial  production,  such  as  the  current  GAO 
investigation  of  commonality  problems  between 
Navy  and  Air  Force  radar  warning  receivers. 

6.  Organizational  design  and  interaction  of  the 
primary  departments  involved  in  the  SOA 
extension . 

A  key  secondary  objective,  as  discussed  in  Chapter 
Two,  is  to  clearly  delineate  the  history  of  previous 
research  of  states-of-the-art  extension  questions.  The 
fundamental  theory  and  key  elements  of  early  SOA 
measurement  equations  from  1965  to  1985  are  outlined. 
Examples  of  SOA  model  applications  to  aircraft  turbine 
engines,  ground  combat  surveillance  radars,  computers, 
and  laser  weapons  systems  is  presented. 

D.   THE  RESEARCH  QUESTION 

The  primary  research  question   examined   was:    how 
effectively   did   Litton   estimate   the  development  and 
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production  costs  of  the  state-of-the-art  extension  of 
the  AN/ALR-67  Radar  Warning  System?  Subsidiary- 
questions  will  deal  with  Litton's  budgetary  models  for 
predicting  and  controlling  costs.  Their  methods  of 
cost  control  are  also  explored. 

Three  subsidiary  questions  were: 

1.  What  previous  research  has  been  conducted  on 
weapon  system's  state-of-the-art  extensions? 

2.  Which  cost  estimating  relationships  were 
utilized  in  Litton's  cost  estimation  of  the 
development  program? 

3.  Was  there  an  association  between  levels  of  SOA 
extension  and  cost  overruns? 

E.   SCOPE  AND  LIMITATIONS 

This  thesis  concentrates  on  two  specific  areas. 
First,  Litton's  cost  estimation  process  in  measuring 
the  state-of-the-art  advance  from  the  AN/ALR-45  to  the 
AN/ALR-67  Radar  Warning  System  is  thoroughly  discussed. 
Second,  the  SOA  measurement  theory,  control  and 
planning  techniques  for  defense  contractors  involved  in 
research  and  development  programs  is  analyzed.  For 
example,  the  general  nature  of  the  S-curve  is 
described.  The  S-curve  represents  how  technology 
initially  advances  slowly,  then  rapidly  gains  momentum 
and  eventually  slows  down  when  it  nears  the  natural 
limits  of  the  technology.  [Ref .  6]  Contractors  must 
be  able  to  accurately  locate  their  exact  position  along 
the  curve,  to  determine  applicability  of  SOA 
extensions . 

In  summary,  the  case  study  describes,  then  assesses 
Litton's  ability  to  accurately  predict  and  control 
development  and  production  costs  for  the  AN/ALR-67 
Radar   Warning    System.     Peripheral   information   is 
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provided  as   review   factors   in   the   organization  and 
budget  process  which  contribute  to  the  firm's  success. 

The  principal  limitation  of  this  case  study  is  that 
there  is  no  guarantee  that  observations  are 
transferable  to  another  environment,  setting,  project, 
or  firm.  For  example,  the  unique  mission  of  fighter 
aircraft  radar  warning  systems  may  negate  the  transfer 
ability  of  finds  to  commercial  aircraft  manufacturing 
firms.  Also,  one  sample  may  not  be  truly 
representative  of  shipbuilding  projects,  the  primary 
focus  of  Litton's  other  defense  divisions. 

Litton  Applied  Technology  placed  restrictions  on 
availability  of  the  exact  algorithms  utilized  in  their 
cost-estimation  model.  Specific  cost  data  which  might 
reveal  overhead  rates  or  management  reserve  factors  to 
their  competitors  was  withheld.  Also,  the  cost 
estimator  primarily  responsible  for  initial  development 
costs  during  the  1975-79  time  frame  was  not  available 
for  interviews.  The  rationale  behind  the  initial  cost 
estimates  was  not  clearly  documented,  so  many 
observations  of  the  SOA  extension  will  be  based  on 
secondary  sources.  Additionally,  due  to  the 
unclassified  nature  of  this  thesis,  many  current 
mission  and  performance  parameters  of  the  AN/ALR-67 
Radar  Warning  System  are  omitted. 

F.   LITERATURE  REVIEW  AND  METHODOLOGY 

Extensive  research  has  been  conducted  on  methods  of 
cost  estimation  for  weapons  systems.  The  majority  of 
the  literature  is  based  on  parametric  models,  both  for 
hardware  and  software  cost  estimation.  Significantly 
less  literature,  however,  is  available  on  measurement 
of  state-of-the-art  advances.  Knowledge  about  the 
critical  process   of   high   technology   cost  estimation 
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will  enable  a  more  credible  analysis  of  Litton's  cost 
estimation  procedures  for  the  AN/ALR-67  Radar  Warning 
System.  This  case  study  can  then  explore  the  extent  to 
which  theoretical  research  from  the  past  two  decades  is 
practically  utilized,  if  at  all,  by  one  of  our 
country's  leading  defense  electronics  manufacturers. 

For  SOA  extensions  background  literature,  the 
following  key  authors  and  their  specialties  are 
highl ighted : 

1.  A. A.  Alexander:  estimation  of  advanced  technology 
in  turbine  engines. 

2.  E.N.  Dodson:   quantitative  measurement  of  state- 
of-the-art  through  use  of  planar  and  ellipsoid 
surfaces . 

3.  Richard  Foster:   analysis  of  the  S-curve,  or 
Gompertz  curve,  characteristics. 

4.  Results  from  the  1983  Technological  Forecasting 
Conference  which  focused  on  this  area. 

The  methodology  for  this  thesis  primarily  was  a 
case  study  of  Litton  Applied  Technology,  Inc.,  in  San 
Jose,  California.  Specifically,  personal  interviews 
were  conducted  over  a  period  of  two  months  with  the  top 
managers  involved  in  the  AN/ALR-67  development.  Among 
the  key  company  personnel  interviewed  were  Eugene  E. 
Deimling,  Director  of  Business  Development,  and  Donald 
R.  Bowden,  Director  of  Cost  Estimating  and  Analysis. 
Personal  interviews  were  also  conducted  with  Dr.  Edward 
Dodson,  Director,  Economic  Resources  and  Planning  Group 
of  General  Research  Corporation,  Santa  Barbara, 
California,  and  Stan  Swales,  Cost  Estimator  for  GTE 
Government  Systems,  Inc.,  Mountain  View,  California. 

Searches  through  the  Naval  Postgraduate  School 
Library  and  the  Defense  Logistics  Systems  Logistics 
Exchange  were  also   conducted.     The   primary  journals 
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investigated  were  "Research  Management",  "IEEE 
Transactions  on  Engineering  Management",  and 
"Technological  Forecasting  and  Social  Change".  Also, 
current  articles  ordered  from  the  librarian  of  the 
Space  Systems  Cost  Analysts  Group  and  the  International 
Society  of  Parametric  Analysts,  Mr.  Clyde  Perry,  were 
reviewed . 

Finally,  the  Litton  Pricing  and  Cost  Estimation 
Manual  was  examined,  with  particular  emphasis  on 
procedures  for  state-of-the-art  developments. 

G.   ORGANIZATION  OF  STUDY 

The  remaining  chapters  of  this  thesis  are  organized 
as  follows: 

1.  Chapter  Two:  Literature  review  of  previous 
studies  of  state-of-the-extensions  measurement 
theory  is  presented.  Also,  comparisons  between 
various  hardware  and  software  cost  estimating 
models  are  made.  Mechanisms  espoused  in  current 
literature  for  controlling  high  technology 
development  projects  is  briefly  outlined. 

2.  Chapter  Three:  A  historical  narrative  of  two 
cost  estimation  models  by  Dr.  E.N.  Dodson  is 
presented.  Examples  will  depict  cost  elements 
which  have  factored  into  them  SOA  technological 
factors . 

3.  Chapter  Four:  The  actual  cost  data  from  the 
AN/ALR-67  is  presented,  along  with  a  description 
of  Litton's  pricing  and  cost  estimation  system. 
Interviews  with  key  individuals  are  addressed. 

4.  Chapter  Five:  The  analysis  of  variances  found  in 
the  cost  data,  and  external  events  which  have 
impacted   Litton's   development   of  the  AN/ALR-67 
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are  discussed.   Comparisons   from  the   case  study 

to  relevant  theory  are  made. 

Chapter   Six:     Conclusions   and  recommendations 

developed   in   this   case    study   are   concisely 

stated.       Areas    for    future    research   are 

identified. 


16 


II.   LITERATURE  REVIEW 


A.   INTRODUCTION 

This  chapter  explores  the  background  literature 
pertinent  to  measurement  and  control  of  state-of-the- 
art  advances.  In  particular,  a  chronological  outline 
of  SOA  measurement  history  from  1969  to  1985  is 
presented.  The  chapter  is  subdivided  into  the 
following  key  areas: 

1.  Ostwald's  basic  guidelines  for  measuring 
technological     advances  and  cost  estimating 
relationships . 

2.  Dodson's  approach  to  quantitative   measurement  of 
advances  in  state-of-the-art  in  January  1969. 

3.  Dodson's  studies   in  resource  estimation  for 
development  programs  from  October  1969. 

4.  Rand  Corporation's  study  of  measurement  of 
technological  change   in  aircraft  turbine  engines 
(June  1972). 

5.  Hovanssian's  description  of  key  parameters 
integral  to   research  and  development  of  large- 
scale  electronic  systems  (August  1975). 

6.  Dodson's  development  of  cost  equations  to  measure 
technological  change   in  high-technology  systems 
(May  1977). 

7.  Gordon  and  Munson's  proposed  convention  for 
measuring  the   state-of-the-art  of   products 
(1981  ). 

8.  Alexander  and  Mitchell's  measurement  of 
technological  change   of  heterogeneous  products 
( 1985  )  . 
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9.  Martino's  measurement  of  technology  using 
tradeoff  surfaces  (1985). 

10.  Dodson's   measurement  of  SOA  and  technological 
advance  ( 1985  )  . 

11.  Cooley,  Hehmeyer,and  Sweeney's   model  of  research 
and  development  resource  allocations  (1986). 

12.  McDonough ' s   identification  of  effective 
characteristics  of  management  control   systems  of 
new  product  development  projects  (1984). 

13.  Smith's   summarization  of  best  techniques  for 
estimating  time,  cost,  and  resources  in  new 
developments  (1985). 

14.  Evaluation   of  the  key  hardware  and  software  cost 
estimation  models  used  in  advanced  technology 
development  and  production  contracts. 

B.   OSTWALD'S  BASIC  TOOLS  FOR  COST  ESTIMATING 

Ostwald's  "Cost  Estimating  for  Engineering  and 
Management",  published  in  1974,  provides  the  essential 
basis  for  detailed,  practical  cost  estimates.  [Ref.  7] 
Stan  Swales,  cost  estimator  at  GTE,  recommended 
Ostwald's  book  as  a  prerequisite  to  analyses  of  current 
cost  estimating  techniques.  Ostwald  details  the  first 
unified  treatment  of  the  philosophy,  concepts,  and 
practices  of  the  cost  estimating  field.  The 
fundamentals  of  three  ideas  pertinent  to  later  studies 
of  SOA  advance  are  introduced:  cost  estimating 
relationships  ( CER ' s  )  ,  cost  indices,  and  technological 
forecasting . 

1 .   Cost  Estimating  Relationships 

Ostwald  describes  CER's  as  functional  models 
that  mathematically  describe  the  costs  of  components  as 
functions  of  one  or  more  independent  variables.  CER's 
are  used  to  estimate  physical  quantities  such  as  number 
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of  radars  or  personnel.  CER * s  also  can  express  rates 
of  activity  for  support  personnel  as  a  function  of  the 
number  of  direct  radar  operators,  for  life  cycle  cost 
studies.  In  aerospace  electronic  industries,  CER ' s  can 
be  used  to  estimate  turbojet  engine  development  cost  as 
a  function  of  maximum  thrust  and  production  quantity. 
Three  fundamental  elements  must  be  present  in  CER ' s : 

a.  Logical  relationship  of  the  variable  to  cost, 

b.  Statistical  significance   of  the  variables' 
contribution,    and 

c.  Independence  of  variables   to   the   cost 
explanation . 

2 .  Technological  Forecasting 

Ostwald  defines  technological  forecasting  as 
"logical  analyses  that  leads  to  quantitative 
conclusions  about  future  engineering  qualities  and 
properties."  [Ref.  8]  He  recommends  evaluation  of 
technology  trends  to  find  the  critical  independent 
variables  on  which  the  dependent  variables  rely. 
Correct  analysis  of  the  S  curve  is  an  essential 
ingredient.  The  S  curve  is  a  feature  of  nature  that 
diagrams  how  electronic  components  are  subject  to  flat 
and  slow  growth  in  the  early  years,  rapid  middle-life 
development,  followed  by  stability  and  eventually 
decline  as  new  products  evolve.  General  technical 
history  books  provide  back-up  documentation. 

3 .  Cost  Indices 

Ostwald  also  describes  how  cost  indices 
enable  estimators  to  forecast  the  cost  of  new  designs 
based  on  similar  items  in  the  past,  without  going 
through  detailed  "bottoms-up"  costing.  The  cost  index 
is  a  dimensionless  number  for  a  given  year,  indicating 
the  cost  of  that   year  relative   to  a   base  year.    The 
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shows  the  conversion  from  costs  to   equivalent  costs  in 
the  present. 

C.  DODSON'S  APPROACH  TO  QUANTITATIVE  MEASUREMENT  OF 
ADVANCES  IN  STATE-OF-THE-ART  (JANUARY  1969) 
In  1969,  Dr.  E.N.  Dodson  defined  state-of-the-art 
in  concept  as  the  highest  degree  of  technical 
accomplishment  that  could  be  achieved  at  any  point  in 
time.  [Ref.  9]  This  was  later  revised  to  represent  the 
state  of  best  implemented  technology.  This  early  study 
focused  on  parameters  which  comprehensively  influenced 
achievable  engineering  designs. 

Dr.  Dodson  stated  that  the  following  constraints 
must  be  met  before  SOA  advances  could  be  quantitatively 
measured : 

1.  A  geometrical  surface  with  continuous  derivatives 
that  interpolates   between  implemented  design 
characteristics  is  used  to  approximate  the   true 
state-of-the-art. 

2.  The  surface  should  be  either  convex  or  concave  in 
all  dimensions,  permitting  plane  surfaces. 

3.  Each  pair  of  design  characteristics  must  be 
negatively  correlated   to  allow  tradeoffs  on 
single  state-of-the-art  surfaces. 

Considering  these  constraints,  Dr.  Dodson  asserted 
that  a  convex  hypersurface  reasonably  represented  SOA 
surfaces.  Actual  data  points  are  fitted  to  convex 
surfaces  in  ellipsoid  geometric  forms  via  mathematical 
relationships.  The  chosen  physical  or  performance 
characteristics  must  be  among  those  derived  early  in 
the  concept  exploration  acquisition  stage.  These 
characteristics  should  be  influenced  by  engineering 
development  decisions  to  have  relevance  to  SOA 
determination.      Also,   Dr.   Dodson   stipulated   that 
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characteristics  should  be  specified  so  that  increasing 
values  corresponded  with  increasing  technical 
difficulty. 

Dr.  Dodson  worked  with  subsystems  of  solid 
propellant  missiles,  maintaining  the  Department  of 
Defense  Work  Breakdown  Structure.  Three  parameters  of 
propulsion  subsystems  which  met  his  mandatory 
prerequisites  were:  delivered  specific  impulse, 
propellant  mass  fraction,  and  length-to-diameter  ratio. 
These  parameters  were  considered  the  primary,  but  not 
all  inclusive,  factors  which  represented  the 
technological  advance.  These  variables  influenced 
areas  of  SOA  relevance  such  as  variations  in  chamber 
pressure  and  burning  rate.  Dr.  Dodson  then  assigned 
values  to  these  parameters  based  on  the  Chemical 
Propulsion  Information  Agency  Rocket  Motor  Manual.  His 
next  action  was  to  classify  these  motors  in  various 
time  periods  according  to  year  of  development 
completion . 

Dr.  Dodson 's  purpose  in  quantifying  measurements  of 
advances  in  the  SOA  was  to  test  the  hypothesis  that  SOA 
extensions  influenced  the  costs  required  for 
development  programs.  Through  geometric  methods,  he 
based  SOA  measurement  advances  upon  how  data  points  fit 
above  existing  SOA  surfaces.  The  data  points 
represented  the  chosen  parameters,  classified  by  year 
of  development.  The  measured  SOA  advance  becomes  a 
function  of  the  proportionate  increase  in  the  radial 
drawn  from  the  origin  through  the  surface  to  the  new 
data  point.  The  SOA  advance  is  indicated  by  the 
squared  proportional  increase. 

Dr.  Dodson  summed  up  his  1969  study  by  stating  that 
the  measured  SOA  advance  is  comprised  of  these  two 
factors : 
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1.  Attributes  of   design  efforts  directly  related  to 
the  particular  system  development  program,  and 

2.  Contributions  from  general  research  and 
development  factors,   which  can  be  represented  by 
dummy  variables  identified  for  specified  major 
technological  advances. 

D.   DODSON'S    STUDY     OF    RESOURCE    ESTIMATION    IN 

DEVELOPMENT  PROJECTS 

Dr.  Dodson,  in  a  study  for  the  Assistant  Secretary 
of  Defense,  elaborated  his  earlier  study  on  SOA 
advance.  He  described  the  SOA  for  a  particular  point 
in  time,  with  n  representing  the  number  of  SOA  design 
characteristics.  These  characteristics  could  trade  off 
upon  one  another.  Some  examples  are  weight,  speed,  and 
size  for  fighter  aircraft's  SOA. 

Dr.  Dodson  indicated  that  Work  Breakdown  Structures 
should  be  used  to  divide  whole  weapons  systems,  such  as 
missiles,  into  subsystems.  SOA  specification  for  an 
aggregate  system  would  not  be  reliable  since  subsystem 
technologies  are  distinct  and  will  advance 
independently  of  one  another.  For  example,  missile 
propulsion  and  guidance  subsystems  evolved  separately, 
with  different  primary  parameters.  He  recognized  that 
information  about  performance,  development  costs,  and 
development  time  of  subsystem  is  severely  limited,  due 
to  contractor  reluctance  to  divulge  this  information. 

In  this  report,  Dr.  Dodson  concentrated  on 
specifying  SOA  measurement  equations  for  inertial 
guidance  systems  for  missiles.  Accuracy,  weight,  and 
reliability  were  selected  as  the  principal  parameters 
affecting  SOA.  These  three  factors  could  be 
quantified.     They   explained,   based   on   engineering 
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judgment,  most  of  the  resources  spent  on  advancing  the 
SOA. 

The  measure  of  SOA  for  accuracy  was  maximum  range 
divided  by  Circular  Error  Probable  {CEP}  for  inertial 
guidance  systems.  CEP  represented  the  acceptable 
distance  a  missile  must  steer  to  its  target.  This 
distance  could  be  fractions  of  miles  for  a  nuclear- 
armed  ballistic  missile.  Since  state-of-the-art 
advance  always  increases,  the  above  equation  indicated 
that,  as  range  increased,  given  accuracy  became  more 
difficult  to  achieve. 

The  measure  of  SOA  for  weight  was  the  number  one 
divided  by  the  guidance  system  weight.  Dr.  Dodson's 
study  found  light  guidance  systems  outperformed  heavy 
ones  since  they  permitted  greater  range  and  payload. 

The  SOA  parameter  for  reliability  was  mean-time- 
between-f ai lure  (MTBF).  For  inertial  guidance  systems, 
reliability  was  critical  due  to  the  continuous  state  of 
twenty-four  hours  a  day  alert  status. 

Dr.  Dodson  indicated  several  reasons  why  production 
cost  was  not  a  good  choice  for  an  SOA  parameter: 

1.  Costs  prior  to  development  would  be  derived  from 
cost  estimating  relationships,  which  would  not  be 
historically  reliable. 

2.  Learning  curves  could  not  be  established  before 
actual  engineering  efforts  had  been  expanded  to 
reduce  costs. 

Finally,  by  geometric  measures  of  the  data  points 
for  accuracy,  weight,  and  reliability  on  an  ellipsoid 
surface,  SOA  advance  indices  were  developed.  Some  SOA 
index  numbers  were  2.945  for  the  Pershing  missile,  and 
120.200  for  the  Polaris  missile.  Research  using  the 
SOA  parameters  demonstrated  Pershing  had  a  relatively 
low  SOA  advance  for  two  reasons: 
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1.  Relatively  low  accuracy  at  maximum  range,  and 

2.  Low  reliability. 

Finally,  Dr.  Dodson  devised  a  logarithmic  approach 
to  measure  the  cost  of  an  SOA  advance,  in  relation  to 
time.  His  basic  hypothesis  stated  that  time  and  cost 
are  interrelated  resources.  He  developed  an  equation 
treating  the  cumulative  cost  to  complete  development 
(C)  as  a  dependent  variable,  with  the  measured  advance 
in  SOA  (S)  as  an  independent  variable.  The  elapsed 
time  to  development  completion  (T)  would  be  stipulated 
by  the  planner.   The  equation  took  the  form  of: 

log  C  =  a  +  b  log  S  +  C  log  T, 

where  a,b,  and  c  were  designated  parameters,  based 
on  the  system. 

The  next  section  outlines  how  Rand  Corporation 
expanded  on  Dr.  Dodson 's  approach. 

E.   MEASURING  TECHNOLOGICAL   CHANGE  IN  AIRCRAFT  TURBINE 

ENGINES 

Alexander  and  Nelson  from  Rand  Corporation 
developed  techniques  in  1972  to  measure  technological 
advance  in  weapons  acquisition.  [Ref.  10]  Their 
initial  studies  focused  on  aircraft  turbine  engines. 
Their  goal  was  to  capture  mainstream  trends  and  improve 
estimates  of  costs  and  schedules  during  the  system 
acquisition  cycle. 

Alexander  and  Nelson  used  multiple  regression 
analysis  to  develop  an  equation  containing  the  primary 
parameters  important  in  turbine  engine  technological 
change.  Like  Dodson  previously,  they  estimated 
multidimensional  tradeoff  surfaces  of  the  key  turbine 
engine  parameters.  Using  regression  analysis,  they 
traced  out  the  movement  of  the  tradeoff  surface  over 
time  . 
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The  equation  they  developed  to  calculate  indices 
for  turbine  engine  development  was: 

Tech  =  -1187.5  +  156   Ln   Temp   +   18.8   Ln  Thrust- 

26.5  Ln   Weight  -   20.6  Ln   SFC  +   11.7  Ln  Q  +  13.0 

Prop . 

The  dependent  variable  represented  the  technology 
index,  which  was  measured  in  quarters  of  a  year 
beginning  in  January  1943.  The  independent  variables 
represented  were: 

1.  Temp  =   turbine  inlet  temperature  in  degrees 
Rankine . 

2.  Thrust  =  military  sea  level  static  thrust  in 
pounds . 

3.  Weight  =  engine  weight  in  pounds. 

4.  SFC  =   specific  fuel  consumption  at  military  sea 
level  staticthrust  (lb/hr/lb). 

5.  Q  =  maximum  dynamic  pressure  in  pounds  per  square 
foot . 

6.  Prop  =  dummy  variable,  equal  to  one  if  the  engine 
was  a  turboshaft  or  turboprop,  zero  or  otherwise. 

Alexander  and  Nelson  found  that  the  major  turbine 
engine  manufacturers  had  similar  tradeoff  surface 
shapes.  However,  the  two  major  manufacturers,  General 
Electric  and  Pratt  and  Whitney,  were  approximately  two 
years  ahead  of  their  competitors  in  level  of 
technology . 

Alexander  and  Nelson  specifically  addressed  the 
question:  "Can  a  technique  be  developed  for  objectively 
quantifying  the  technological  state  of  the  art  of  a 
particular  type  of  system?"  [Ref.  11]  Their  analysis 
utilized  the  following  two  assumptions: 

1.   Limited  numbers   of  parameters   adequately 
characterized  the  system  under  study. 
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2.   Historical  continuity  prevailed  such  that  the 

selected  parameters  characterized  the  system  even 
during  different  time  periods. 

Alexander  and  Nelson  also  recommended  the  use  of 
performance  parameters  rather  than  technical 
parameters,  to  give  greater  emphasis  to  outputs  than 
inputs.  However,  they  acknowledged  their  final 
equation  contained  both  technical  and  performance 
parameters.  Their  data  used  in  their  study  came  from 
engine  manufacturers,  and  standard  sources  such  as 
"Jane's  All  the  World's  Aircraft."  Turbine  engines 
were  well-suited  for  analysis  due  to  their  strong 
technological  trends,  such  as  the  increase  in  aircraft 
speeds  and  the  progression  of  engine  types. 

Alexander  and  Nelson  subjected  the  data  to 
statistical  tests  with  different  subdivisions  to  find 
the  equation  with  the  best  "fit"  over  the  tradeoff 
surface.  However,  they  did  not  develop  specific 
equations  for  obtaining  the  cost  of  development  of  the 
technological      advance.  They     hypothesized 

quantification  of  the  technological  setting  were 
necessary  prerequisites,  before  the  cost  question  could 
be  answered. 

In  the  next  section,  Hovassian's  Key  Parameters 
necessary  for  development  of  electronic  systems  are 
examined . 

F.   HOVANSSIAN'S  DESCRIPTION  OF  KEY  PARAMETERS 

Hovanssian  described  the  research  and  development 
phase  of  a  hypothetical  large-scale  electronic  system, 
based  on  his  study  of  applicable  programs.  [Ref.  12] 
He  stated  that  the  initial  design  phase  should  always 
include  a  state-of-the-art  analysis  of  the  key  system 
design  parameters.   This  analysis  should  include  a  risk 
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evaluation  of  potential  problem  areas  such  as 
technology,  cost,  or  time.  He  stated  that  the  more 
current  performance  measures  are  advanced,  the  greater 
the  risk  involved. 

Hovanssian  did  not  develop  any  models  to  estimate 
the  SOA  advance.  However,  many  of  his  cost  estimating 
techniques  matched  those  of  Litton  Applied  Technology. 
Litton's  cost  model  is  described  in  Chapter  Four. 

Hovanssian  stated  that  the  cost-estimating 
relationship  technique  should  only  be  used  during  early 
stages  of  development.  He  recommended  that  the  cost- 
estimating  relationships  be  based  on  statistical 
analysis  of  similar  equipment.  If  statistical  data  on 
similar  equipment  was  not  available,  then  he 
recommended  the  utilization  of  "bottoms-up"  system- 
level  cost  estimating.  In  systems-level  estimating, 
costs  are  derived  from  consideration  of  manpower,  basic 
units,  components,  parts,  or  other  relevant  factors. 

Hovanssian  also  described  "design-to-cost" 
modeling.  In  this  technique,  various  design 
configurations  are  tested  against  specified  performance 
parameters.  The  successful  design  configurations  are 
traded  off  to  determine  which  yields  the  lowest  life- 
cycle  cost.  Life-cycle  costs  included  both  the  total 
procurement  cost  of  the  system  and  its  future  operating 
costs . 

Finally,  Hovanssian  stated  that  most  electronic 
systems  must  include  customer-acceptance  parameters  at 
the  development  stage,  or  face  possible  cost  overruns 
later  in  the  life-cycle.  Among  the  parameters  listed 
were  : 

1.  Life  cycle  cost. 

2.  Maintenance  skill  required  to  repair  the  system. 
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3.  Number  of  maintenance  employees   required  to  keep 
the  system  in  working  order. 

4.  Percentage  of   time  the  system  is  available  for 
use . 

5.  Reliability  (measure  elapsed  time  between  two 
consecutive  failures). 

6.  Amount  of   maintenance  required  per  operating 
hour  . 

7.  Quantity  and  type  of  spare  parts  required. 

8.  Operator  approval  of  new  system. 

9.  Degree  of  automation. 

10.  Improvement  over  previous  systems. 

In  summary,  Hovanssian's  paper  pointed  out  many 
parameters  and  fundamental  concepts  which  entered  into 
the  design  and  development  process  for  large  electronic 
systems.  In  the  next  section,  Dr.  Dodson's  updated 
1977  study  on  quantification  of  SOA  advance  is 
outl ined . 

G.   DODSON'S  DEVELOPMENT   OF  COST   EQUATIONS  TO  MEASURE 

TECHNOLOGICAL  CHANGE 

In  May  1977,  Dr.  Dodson  outlined  procedures  to 
account  for  technological  change  via  research  and 
development  cost  equations.  He  indicated  research  and 
development  costs  were  directly  influenced  by  the 
degree  of  extension  to  the  SOA  by  a  given  program. 
[Ref.  13]  His  measurement  approach  to  the  SOA  advance 
evolved  from  a  combination  of  his  earlier  study 
(discussed  in  section  C)  and  Alexander  and  Nelson's 
paper  (discussed  in  section  D).  Dr.  Dodson  felt  this 
new  approach  would  be  easier  to  implement. 

This  approach  featured  four  key  elements: 
1.   Independent   variables   for  a  multiple  regression 
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equation   would  consist  of  a   limited  number   of 
key  technical  parameters. 

2.  The  dependent   variable  would  be  a  consistently- 
defined  calendar  milestone.   For  example,  a 
completion  of   qualification     test  would 
represent  this  milestone  for  applicable 
subsystems . 

3.  An  expected   date  (Ye)  for  achievement,  for  the 
parameters  designated  as  independent   variables, 
would  be   the  result  of  a      multiple  regression 
exercise . 

4.  The  residuals  of  the  difference  between  the 
expected  end   the  actual  date  of  achievement 

(Y  estimated  -  Y  actual),  would  be  the  measure  of 

the  relative  technological  advance. 
Dr.   Dodson   illustrated   his   theory   through   his 
previous   research    on    avionics    computers.      The 
computers'  key   parameters  were  selected  from  technical 
data  as: 

1.  The  speed  of  the  computer  in  operations  per 
second . 

2.  The  density   of  the  central  processing  unit  in 
pounds  per  cubic  foot. 

3.  The  number  of  distinct  instruction  types   in  the 
computer . 

Dr.  Dodson  also  documented  the  actual  date  of 
development  completion  for  each  computer  as  Y  actual. 
The  difference  between  the  expected  and  actual  dates  of 
completion  represented  the  change  in  SOA ,  in  units  of 
years . 

Once  the  residual  was  calculated,  it  was 
incorporated  into  a  cost  estimating  relationship.  The 
dependent  variable  would  represent  the  development  cost 
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associated  with  the  specific  SOA  advance.  The 
independent  variables  were: 

1.  XI,  which   represented  the  residual  difference 
between  the  expected  and  actual  dates  of 
development  completion. 

2.  X2 ,  which   equalled  one  for  microprogrammable 
computers,  or  zero  for  synchronous  computers. 

3.  X3 ,  which  equalled  one  for  space  computers,  or 
zero  for  airborne  computers. 

The  resulting  equation  developed  by  Dr.  Dodson  to 
calculate  the  monetary  amount  correlated  with  SOA 
advance  was: 

Y  =  6.11  +  2.7  XI  -  4.57  X2  +  14.8  X3 

Dr.  Dodson  recognized  several  limitations  to  his 
SOA  measurement  approach.  The  equation  might  simply 
reflect  a  change  of  mission  requirements,  rather  than 
indicate  technological  advance.  To  remedy  this 
potential  problem,  he  recommended  that  variables  should 
be  stated  via  efficiency  measures  like  thrust  per 
pound,  rather  than  absolute-scale  measures  like  thrust. 

Dr.  Dodson  also  recommended  in  this  paper  that 
electronics  development  firms  should  develop  an  output 
index  to  account  for  year-to-year  changes  in 
manufacturing  productivity.  He  felt  that  failure  to 
consider  productivity  changes  contributed  to  errors  in 
estimating  electronics  procurement  costs.  Two  reasons 
for  productivity  changes  were: 

1.  Changes  in   design  and  manufacturing  technology 
which  reduce  the  cost  per  unit  of  output. 

2.  Operational  requirements  for  performance 
increases . 

Using  costs  as  the  dependent  variable,  cost 
estimating  relationships  were  derived  by  Dr.  Dodson  for 
the  technology   output  indices.   In  this  case,  the  year 


30 


of  development  completion  became  an  independent 
variable.  The  regression  coefficient  represented  the 
annual  change  in  costs  due  to  shifts  in  the 
relationship  between  productivity  and  cost. 

In  summary,  Dr.  Dodson  developed  a  method  to 
measure  the  financial  impact  of  SOA  advance.  In  the 
next  section,  Gordon  and  Munson  proposed  an  alternate 
method  for  measuring  the  SOA  advance. 

H.   GORDON   AND    MUNSON' S    PROPOSED    CONVENTION   FOR 

MEASURING  THE  SOA  OF  PRODUCTS 

In  1981,  Gordon  and  Munson  developed  for  the 
National  Science  Foundation  a  general  equation  to 
measure  SOA  advance.  [Ref.  14]  The  equation  derived 
was  : 

SOA  -  Kl  (P1/P2')  +  K2  (P2/P2')  +  ...  Kn  (Pn/Pn'). 
In  the  above  equation  Kn  represented  the  relative 
weight  associated  with  each  parameter  describing  the 
technology.  Pn  indicated  the  value  of  the  parameter 
chosen  to  describe  the  SOA,  while  Pn '  represented  the 
maximum  reference  value  of  the  parameter. 

Gordon  and  Munson  developed  their  SOA  equation  to 
satisfy  the  following  criteria: 

1.  The  same   estimate  of  SOA  should  be  produced  by 
any  analyst  studying  the  same  technology. 

2.  The  SOA  value  should  be  in  an  index  format,  based 
on  a  reference  value. 

3.  The  equation   should  satisfy   any  level  of 
technological  aggregation. 

Gordon  and  Munson  discussed  two  techniques  critical 
to  the  choice  of  parameters  and  their  relative  weights: 
Judgmental  and  statistical  techniques.  They 
recommended  Delphi-related  procedures,  the  utilization 
of  experts  to  choose  the  proper  parameters  for   the  SOA 


31 


equation.  Delphi  procedures  sought  expert  judgment  via 
anonymity  and  controlled  feedback,  to  obtain  the  most 
objective  inputs.  For  example,  electronic  voting 
devices  were  utilized  to  allow  expert  practitioners 
freedom  to  provide  anonymous  answers  to  questions  posed 
by  group  moderators. 

Gordon  and  Munson  also  listed  several  statistical 
methods  available  to  estimate  weights  and  parameters  of 
the  SOA  equation.  The  techniques  included:  multiple 
regression,  stepwise  regression,  discriminate  analysis, 
cluster  analysis,  multidimensional  scaling,  and  factor 
analysis.  In  factor  analysis,  for  example,  a 
correlation  matrix  was  utilized  to  group  variables  into 
classes  of  highly  correlated  components.  These  classes 
were  taken  as  parameters  through  construction  of  an 
index  from  individual  members  of  the  class. 

Gordon  and  Munson  applied  their  theory  against 
antibiotics  and  computers.  They  formulated  a 
performance  index  for  computers  based  on  the  judgments 
of  an  expert  panel  using  the  Delphi  approach.  The 
following  three  parameters  were  chosen  to  characterize 
computers : 

1.  Computer  speed  by  operations  per  second. 

2.  Cost  of  computation  by  operations  per  dollar. 

3.  Maximum  memory  size  by  kilobytes. 

Gordon  and  Munson  used  computerized  statistical 
techniques,  based  on  their  assumption  that  the  state- 
of-the-art  function  is  an  S-shaped  curve.  Using  their 
results,  they  developed  an  SOA  index  for  computers, 
ranking  the  IBM  3033  computer  as  the  SOA  leader. 

Gordon  and  Munson  felt  their  process  of  calculating 
SOA  measures  could  be  applicable  to  any  field, 
including   integrated   circuits.    In  the  next  section, 
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Alexander  and  Mitchell's  study   of  technological  change 
of  heterogeneous  products  is  outlined. 

I.   ALEXANDER    AND    MITCHELL'S    MEASUREMENT    OF 

TECHNOLOGICAL  CHANGE 

In  1985,  Alexander  and  Mitchell  developed 
procedures  to  measure  technological  change  of 
heterogeneous  products.  [Ref.  15]  They  designed  a 
framework  in  which  to  place  empirical  measures  of 
technological  change,  hedonic  price  indices,  and  cost 
estimating  relationships.  In  this  study  they  estimated 
technological  change  equations  for  milling  machines, 
turbine-powered  airliners,  and  turbine  engines.  Like 
Dodson  earlier,  they  realized  product  characteristics 
alone  could  not  define  technological  change. 

Alexander  and  Mitchell  derived  a  tradeoff  function, 
based  on  a  tradeoff  surface  of  cost,  performance, 
technical  characteristics  and  time.  The  entire 
tradeoff  surface  represented  the  state-of-the-art  of  a 
given  period.   The  tradeoff  function  was  written  as: 

C  =  C  (PI,  . . . ,  Ps,  ql,  . . . ,  qj .  t ). 
The  variables  in  this  function  were: 

1.  "C"  =  average  cost  of  the  product. 

2.  "P"  =   performance  characteristics   or  user 
outputs. 

3.  "q"  =  factor  prices  (assumed  to  be  fixed). 

4.  "t"  =  time. 

Their   equation   meant    technological    change   in 
products  arose  from  three  factors: 

1.  Productivity  or  performance  improvements. 

2.  Improved  factor  inputs. 

3.  Production  process  improvements. 

Alexander   and   Mitchell   noted  that  their  equation 
took   the   form    of    the    typical    cost   estimating 
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relationship,  with  cost  as  the  dependent  variable.  The 
dimensionality  of  the  state-of-the-art  surface  was 
limited  by  the  difference  between  the  total  number  of 
variables  and  the  number  of  constraints.  Their  most 
important  reason  for  including  costs  in  the  measure  of 
technological  change  was  their  belief  that  higher 
levels  of  performance  were  attained  through  the 
expenditure  of  more  resources. 

Alexander  and  Mitchell  studied  the  technological 
histories  of  milling  machines,  airframes,  and  aircraft 
turbine  engines,  summarizing  their  specific  results  in 
this  paper.  They  concluded  that  measures  based  on 
product  characteristics  must  be  evaluated  carefully, 
since  selection  of  a  few  core  characteristics  often 
neglects  other  attributes  whose  relative  importance  my 
have  changed  over  time.  They  felt  productivity 
measures  based  on  user  outputs,  such  as  cost  per  mile 
for  aircraft,  were  more  likely  to  capture  the  totality 
of  technological  change. 

In  the  next  section,  Martino's  studies,  which  also 
concentrated  on  measurement  of  SOA  surfaces,  are 
highl ighted . 

J.   MARTINO'S  MEASUREMENT  OF  TECHNOLOGY   USING  TRADEOFF 

SURFACES 

In  1985,  Joseph  Martino  followed  up  Alexander's 
theoretical  derivation  of  the  designer's  tradeoff 
surface  with  an  empirical  approach.  He  concentrated 
only  on  those  technical  parameters  which  were  relevant 
to  measuring  the  SOA.  He  viewed  the  SOA  as  a  "surface 
in  some  multidimensional  parameter  space."  [Ref.  16] 

Martino  stated  that  an  SOA  advance  was  a 
prerequisite  before  designers  could  move  to  higher 
surfaces.   He  reasoned  that  designers   were  constrained 
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by  technology  and  economics  to  stay  on  the  SOA  surface. 
As  a  result,  improving  the  value  of  one  technical 
parameter,  such  as  speed,  meant  sacrifice  of  another 
parameter,  such  as  weight.  He  examined  several 
technologies  to  see  if  the  data  values  on  a  surface  had 
a  discernible  or  random  pattern.  His  goal  was  to 
determine  the  actual  variables  involved  in  design 
tradeoffs . 

Martino's  theories  drew  heavily  from  Dodson's  work 
except  for  two  statistical  differences: 

1.  Martino  extended  his  method  to  allow   ellipsoid 
surfaces  of  any  order,  not  just  level  two. 

2.  Martino's  surface  fitting  procedure  was  changed 
to  minimize  the  Mean  Absolute  Deviation  rather 
than  Mean  Square  Deviation. 

He  felt  Dodson's  fitting  method  was  prejudiced  by 
extreme  date  values. 

Martino  applied  his  surface-fitting  procedures  to 
clipper  sailing  ships,  jet  engines,  propeller-driven 
aircraft,  and  power  transistors.  In  selection  of 
variables,  he  avoided  the  problem  of  scale  effects  by 
using  nondimensional  variables.  For  example,  he 
divided  a  size  variable  by  a  "characteristic  length"  to 
cancel  out  scale  effects.  He  also  utilized  "data 
triplets",  where  two  positively  correlated  variables 
negatively  correlated  with  a  third  variable. 

Martino's  primary  results  were  that: 

1.  Deviations  from  the  SOA  surface  could  be 
explained  quantitatively   by  known 
characteristics. 

2.  Different  configurations  representing  the   same 
SOA  belonged  on  the  same  surface. 

3.  An  engineering    analysis  was  necessary  in  order 
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to  select  the  proper  variables.    Designers  were 
asked  what  parameters  they  worked  on  to  improve. 
4.   The  tradeoff   surface  technique  worked  for  all 
levels  of  the  Work  Breakdown  Structure. 
In  the   next  section,   Dr.  Dodson's   1985  update  of 
his  SOA  measurement  theory  is  discussed. 

K.   DODSON'S  MEASUREMENT   OF  SOA   AND  TECHNOLOGICAL 

ADVANCE 

Dr.  Dodson's  paper  for  the  Workshop  on  Technology 
Measurement,  Dayton,  Ohio  (12-14  October  1983),  was 
published  in  1985.  [Ref.  17]  This  paper  summarized  his 
earlier  works  from  1969  to  1977.  Also,  he  included  two 
other  approaches  used  to  measure  SOA  advance. 

Dodson  described  the  factor  analysis  approach,  a 
statistical  technique  for  analyzing  variance.  He  felt 
that  factor  analysis  would  help  in  identifying  the 
underlying  relationships  of  the  many  physical  and 
performance  characteristics  of  components.  For 
example,  he  used  factor  analysis  to  rank  sixty  rocket 
motors  by  "Technological  Distance  Scores".  He  gathered 
data  from  the  Chemical  Propulsion  Information  Agency 
Rocket  Manual  for  the  following  nine  variables: 

1.  Delivered  specific  impulse. 

2.  Mass  ratio  (propellant  weight/  motor  weight). 

3.  Length-to-diameter  ratio. 

4.  Reciprocal  of  burn  time. 

5 .  Motor  weight . 

6.  Average  thrust/  burn  time  squared. 

7.  Average  thrust. 

8.  Average  chamber  pressure. 

9.  Date  of  development  completion. 

The  factor  analysis  for  the  nine  variables  was  able 
to   account   for   eighty-three   percent   of   the   total 
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variance  of  these  variables.  From  the  computed 
Technological  Distance  Scores,  Dodson  demonstrated 
graphically  in  this  paper  the  effects  of  changes  in 
mission  requirements  and  technological  capabilities. 
For  example,  the  mission  objectives  for  the  Sprint 
missile  required  fast  burning  motors.  Overall,  this 
missile  represented  a  significant  technological 
advance . 

His  second  new  approach  to  measurement  of  SOA 
focused  on  the  time  available  to  develop  levels  of 
technology.  His  results  revealed  that  for  a  given 
number  of  years,  the  higher  the  desired  technology,  the 
greater  the  risk.  Conversely , the  more  time  available, 
the  lower  the  risk.  However,  Dodson 's  research  has  not 
extended  this  variant  of  the  time  factor  into  specific 
examples  of  SOA  measurement.  His  paper  indicates  that 
application  of  this  feature  to  multiparameter  SOA 
surface  equations  must  be  a  subject  of  future  study. 

L.   MODELING   RESEARCH   AND   DEVELOPMENT   RESOURCE 

ALLOCATION 

Cooley,  Hehmeyer ,  and  Sweeney  developed  the 
Technology  Resource  Allocation  Model  (TRAM)  to  analyze 
the  impact  of  project  selection,  funding,  scheduling, 
technical  risk,  and  staffing  upon  an  organization's 
research  goals.  [Ref.  18]  Cooley's  model  was  designed 
to  answer  what-if  questions  such  as  :  what  performance 
degradation  could  be  expected  if  a  five  year 
development  period  was  shortened  to  three  years?  The 
important  design  considerations  in  this  model  were: 
realism,  model  flexibility,  ease  of  use,  and  output 
format.  TRAM  produced  both  tables  and  graphs  which 
plotted  the  expected  progress  of  the  research  effort  as 
a  function  of  time. 
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TRAM  was  structured  to  allow  variability  by 
analysts  in  the  following  factors: 

1.  Required  funding  by  work  unit. 

2.  Actual  funding  by  work  unit. 

3.  Probability  of  success  by  work  unit. 

4.  Engineer  manager  work  load  by  engineer  manager. 

5.  Contribution  factor  for  each  work  unit. 

6.  Schedule  extension  by  work  unit. 

7.  Performance  objectives  by  work  unit. 

Cooley's  model  included  consideration  of  the  S 
curve  effects.  S  curves  normally  depicted 
transpiration  of  relatively  long  lead  times  before 
significant  results  were  achieved. 

TRAM  enabled  the  researcher  to  quickly  analyze  the 
effects  of  budget  reductions  on  state-of-the-art 
development  projects.  Different  measures  of 
effectiveness  were  generated  by  variances  of  cost, 
schedule,  and  performance  criteria.  The  sensitivity 
analysis  for  TRAM  was  accomplished  through  use  of  the 
DYNAMO  Compiler. 

M.   MANAGEMENT  CONTROL   OF  NEW   PRODUCT  DEVELOPMENT 

PROJECTS 

McDonough  felt  management  control  systems  were  an 
essential  ingredient  in  the  success  of  state-of-the-art 
development  projects.  [Ref.  19]  His  research  found 
that  management  control  directly  impacted  SOA  projects 
in  the  following  ways: 

1.  Accuracy  of  cost  and  duration  estimates. 

2.  Rate  of  progress  of  the  SOA  project. 

3.  Size  of  the  development  budget. 

4.  Quality  of  the  output. 

5.  Competitive  ability  of  the  organization. 
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McDonough's  paper  identified  the  characteristics  of 
effective  management  control  systems.  He  also 
identified  deficiencies  common  to  most  research  and 
development  organizations. 

McDonough  surveyed  twelve  large  new  product 
development  projects.  He  found  four  elements  common  to 
all  SOA  developments.  The  first  element  was  setting 
goals  for  new  product  projects.  He  found  the  primary 
goals  were  cost  budgets  and  schedules.  McDonough's 
results  revealed  that  pressure  from  the  Marketing 
Department  forced  the  majority  of  project  leaders  to 
submit  unrealistic  low  cost  and  duration  estimates.  He 
also  found  product  specifications  were  of  more  concern 
to  top  management  than  budget  or  schedule  overages. 

McDonough's  second  element  was  the  monitoring  of 
project  progress.  Companies  surveyed  used  three 
devices  to  monitor  projects:  written  reports,  formal 
meetings,  and  informal  meetings.  He  found  written 
reports  and  formal  meetings  had  drawbacks  in  timeliness 
and  detail  for  highly  innovative  technology-based 
projects.  The  personal  monitoring  of  projects  by  top 
management  via  informal  methods  was  the  only  way  to 
quickly  remedy  new  problems. 

McDonough's  third  critical  element  was  management's 
response  to  deviations  from  schedules  and  budgets.  He 
found  most  companies  reluctant  to  take  action  on 
technical  issues  of  SOA  projects.  Their  most  common 
solution  was  to  simply  assign  more  engineers,  an  action 
that  results  indicated  rarely  provided  desired  effects. 

McDonough's  fourth  element  for  management  control 
systems  was  incentive  provisions  for  performance.  He 
found  management  rarely  tied  individual  rewards  to  the 
attainment  of  budget  and  schedule  goals. 
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In  summary,  McDonough  stated  the  "management  by- 
walking  around"  was  the  key  to  successfully  controlling 
new  SOA  projects.  He  also  recommended  bonuses  to 
project  managers  who  met  budgets  and  schedules. 

N.   SUMMARIZATION  OF  RULES  FOR  BETTER  ESTIMATES 

Smith's  paper  provided  an  overall  summary  on  the 
research  and  application  of  time,  cost,  and  resource 
estimating.  [Ref .  20]  He  developed  rules  for 
generating  better  estimates  applicable  to  any  project. 
The  following  list  presents  Smith's  key  factors 
necessary  for  the  estimating  process: 

1.  Level  of  Detail:   provide  a  more  detailed 
description  via  increased  levels  of  the  work 
breakdown  structure. 

2.  Precise  knowledge  of  the  task  being  estimated. 

3.  Competency  and  knowledge  of  the  process  being 
estimated . 

4.  Importance  of  the  estimate:   estimates  must 
appreciate  significance. 

5.  Common  units:   all  cost  estimates  should  reflect 
the  same  dimensions. 

6.  Uncertainty:  estimates   should  indicate  measures 
of  the  maximum  possible  error. 

7.  Assumptions  should  be  explicitly  stated. 

8.  Uncontrollable  variables  should  be  incorporated. 
Smith  felt   estimates  could   be  improved   by  use  of 

more  than  one  technique,  such  as  estimating  by  analogy, 
firm  quotes,  handbook  estimating,  parametric 
estimating,  or  regression  analysis.  He  also  stated 
that  relevant  historical  data  was  often  overlooked.  He 
stressed  the  importance  of  detail  in  the  work  breakdown 
structure,  so  errors  in  one  estimate  would  not  have  a 
great    aggregate    effect.      Finally,   Smith   stated 
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management  could  use  certified  professional  estimators 
for  uncertain  projects,  since  they  would  possess 
increased  appreciation  for  the  repercussions  of 
estimates  on  the  baseline  plan. 

0.   EVALUATION  OF  KEY  SOFTWARE  COST  ESTIMATING  MODELS 

In  1981,  Thibodeau  performed  the  Air  Force's  first 
large  scale  study  of  the  various  software  cost 
estimating  models  used  to  make  estimates  of  the 
resources  to  be  invested  in  the  software  subsystems. 
[Ref .  21]  Thibodeau  recognized  that  hardware  cost 
estimating  was  more  advanced,  possessing  more 
identifiable  measures  of  size  and  performance  which  had 
been  correlated  with  cost.  He  found  there  were  no 
reliable  procedures  for  quantitatively  describing  the 
effects  of  non-product  factors  on  cost. 

Thibodeau  evaluated  and  provided  descriptions  for 
the  following  models:  Aerospace,  Boeing,  DOD  Micro 
Estimating,  Doty  Associates,  Tecolote,  Wolverton,  PRICE 
S,  SLIM,  and  Farr  and  Zagorski .  He  provided  one  page 
summaries  for  each  model  type,  including  descriptions 
of  the  estimating  procedure,  characterization  of 
productivity,  and  outputs. 

Thibodeau' s  comparison  of  the  outputs  indicated 

1.  Supporting  materials   for  the  models  did  not 
precisely  state  the  elements  included  in  their 
estimates . 

2.  The  models  were  more  adept  at  satisfying 
information  early   in  the  acquisition  life  cycle. 

3.  The  models  were  acquisition  phase  oriented   and 
did  not  describe  activities  that  crossed 
different  phases. 

4.  Only  PRICE   S  kept  track  of  the  cost  on  a 
component  basis  and  accounted  for  the  cost  of 
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system  integration.    However,  none  of  the  models 
could  provide  costs  for  every  level  called  for  in 
the  Work  Breakdown  Structure. 
Thibodeau    measured    performance    based   on   the 
relative  root  mean  square  error.   He  found: 

1.  Recal ibration  was  the  primary  contributor  to 
differences  in  model  estimating  performance. 

2.  The  structure  of  the  model  was  not  significant  to 
estimating  accuracy. 

3.  The  development  environment  significantly 
influenced  the  performance  of  the  cost  estimating 
models . 

4.  The  use  of  size  as  an  input  had  no  effect  on  the 
relative  performance  of  the  models. 

5.  The  average  root  mean  square  estimating  error  was 
between  fifteen  to  thirty  percent. 

In  his  final  section,  Thibodeau  provided 
recommendations  for  future  model  development  and  better 
data  definition  and  collection.  Finally,  Thibodeau 
described  the  derivation  of  technology  and  complexity 
factors  for  each  of  the  nine  surveyed  cost  estimating 
models . 

P .   SUMMARY 

Chapter  Two  described  a  sampling  of  background 
literature  relevant  to  measurement  and  control  of  SOA 
advance  development  projects.  Chapter  Three  presents 
examples  of  a  few  specific  hardware  cost  estimating 
models  utilized  by  General  Research  Corporation  and  the 
Air  Force. 
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111 •   DEVELOPMENT  OF  COST  ESTIMATING  MODELS 

A.  INTRODUCTION 

Chapter  Three  describes  the  development  of  four 
cost  estimating  models,  including  three  specific  ones 
derived  exclusively  for  weapons  systems.  For  each 
model  the  general  methodology  is  highlighted,  followed 
by  sample  cost  derivations  for  various  cost  elements. 
The  four  models  examined  are: 

1.  Dodson's  cost  estimating  models  for  ground  combat 
surveillance  radars  (1968). 

2.  Dodson's  cost  estimating  methods  for  the 
High-Energy  Laser  Systems  Cost  Model  (1979). 

3.  The  Unmanned  Spacecraft  Cost  Model  (1981). 

4.  The  Freiman  Analysis  of  Systems  Technique  (FAST). 

B.  GROUND  COMBAT  SURVEILLANCE  RADARS 

Dr.  E.N.  Dodson  developed  cost  estimating  methods 
for  ground  combat  surveillance  (GCS)  radars  during  a 
study  for  the  U.S.  Army  Electronics  Command  in  1968. 
The  basic  methodology  is  still  applicable  to  larger, 
more  current  system  studies.  The  goal  of  Dr.  Dodson's 
model  was  to  provide  Army  comptrollers  with  a  model  to 
evaluate  contractor  cost  estimates.   [Ref.  22] 

Dodson's  GCS  radar  model  considered  state  of  the 
art  limitations.  He  stated  that  the  major  objective  in 
the  design  of  GCS  radars  was  to  achieve  minimum  weight 
for  a  specified  performance.  He  used  maximum  range  as 
the  crude  measure  of  radar  performance.  To  prevent 
development  and  production  cost  penalties,  Dodson's 
study  indicated  that  radar  design  engineers  imposed 
constraints  that  radars  to  be  costed  should  not  be 
improved  by  more  than  five  percent  over  the  original 
SOA  curve . 
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To  derive  the  model,  Dodson  first  collected  data  on 
twenty  input  variables.  He  utilized  the  work  breakdown 
structure,  disaggregating  each  life  cycle  element  into 
a  set  of  functional  subsystems.  The  life  cycle 
elements  were  designated  as  engineering  development, 
advanced  production  engineering,  and  production.  The 
work  breakdown  structure  for  engineering  development, 
for  example,  was  broken  down  further  into  hardware 
fabrication  and  documentation. 

In  Dodson's  next  step,  he  derived  a  cost  estimating 
relationship  for  each  category  in  the  work  breakdown 
structure.  The  cost  estimating  functions  were  derived 
through  the  following  basic  steps: 

1.  All  known  factors  between  the  variables  of 
interest  were  specified. 

2.  Regression  equations  were  developed  through  a 
sequence  of   known  intermediate  relationships. 
The  mechanisms  by  which  an  item's  physical 
characteristics  affected  raw  material  quantities 
and  labor  hours  were  investigated.    Engineering 
information  related  the  functional  variables  of 
interest  to  the  physical  configuration. 

3.  Standard  curve  fitting  techniques  determined  the 
constants  associated  with  the  regression 
equations . 

4.  The  statistical  properties  of  the  resulting 
correlation  were  measured. 

Dodson  claimed  the  model's  results  provided  no  more 
than  a  basis  for  judgment,  since  previously  confirmed 
observations  were  his  only  data  source.  Also,  he  did 
not  have  enough  data  to  allow  estimation  of  confidence 
intervals  for  his  cost  predictions. 

During  the  model's  development  stage,  Dodson 
gathered  cost  data  from   seven   different   radars.    He 
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used  a  price  index  developed  at  General  Research 
Corporation  to  normalize  the  dollar  price  data  to  1966 
levels.  His  index  assumed  an  equal  breakdown  of 
engineering  labor,  production  labor,  metals  and 
hardware,  and  electrical  equipment. 

Dodson  utilized  learning  curve  assumptions  to 
normalize  cost  data  with  respect  to  different 
production  quantities.  For  both  complete  radar  sets 
and  subsystems  he  used  a  slope  value  of  ninety  percent. 

The  GCS  cost  model  was  composed  of  the  aggregate 
estimating  relationships  for  each  element.  Dodson 
derived  production  costs  first,  using  the  production 
cost  per  unit  as  an  input  variable  to  the  engineering 
models.  No  more  than  two  or  three  input  variables  were 
required  to  use  any  one  relationship. 

The  model  input  variable  for  production  included: 

1.  Type  of  design,  either  pulsed  or  FMCW. 

2.  Number  of  radars  in  the  first  production  lot. 

3.  Type  of   antenna  material,  either  aluminum  or 
fiber  glass. 

4.  Antenna  frontal  area  in  square  feet. 

5.  Center  frequency  in  megahertz. 

6.  Peak  radiated  power. 

7.  Range  resolution. 

8.  Type  of  presentation,  such  as  handset, 
loudspeaker,  or  plotter  board. 

9.  Total  radar  set  weight. 

10.   Prime  power  source,  either  battery  or 
motor -generated . 
The    only    inputs    required     for    engineering 
development  were  cumulative  average  production  cost  and 
number  of  development  models  fabricated. 

Dodson   derived    advanced   production   engineering 
costs   from   the   cumulative   average   production  cost, 
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number  of  prototype  models  fabricated,  and  the  number 
of  radars  to  be  produced  in  the  first  production  lot. 
The  model  was  built  from  actual  production  cost  data 
describing  the  various  subsystems.  Subsystems  included 
antennas,  scanheads ,  microwaves,  transmitters, 
receivers,  presentation  and  control,  chassis, 
interconnections,  casings,  and  tripods. 

Dodson  derived  a  regression  equation  to  estimate 
the  microwave  subsystem  of  a  radar.  He  defined  the 
microwave  subsystem  as  a  "collection  of  switching  and 
waveguide  components  that  carries  microwave  signals 
between  the  transmitter  and  antenna,  and  the  antenna 
and  receiver."  [Ref.  23]  The  key  input  variables 
researched  were  frequency  and  peak  radiated  power.  His 
final  model  for  microwave  cost  was  derived  by  adding 
the  fixed  waveguide  cost  at  each  frequency  to  the 
power-dependent  duplexer  (switching)  cost.  The  cost 
estimating  relationship  for  pulsed  microwaves  was  found 
through  regression  analysis  to  be: 

$1100  +  $9.4  P,  with  "P"  equal  to  kilowatts. 

Dodson  collected  cost  estimates  for  all  production 
subsystems,  such  as  the  microwave,  to  obtain  total 
subsystem  cost.  Next,  he  added  on  the  final  assembly 
and  test  cost  to  obtain  the  cumulative  average  radar 
set  cost.  The  learning  curve  factor  then  adjusted  the 
total  cumulative  average  radar  set  cost.  The  prime 
power  cost  was  added  to  the  previous  sum  to  obtain  the 
cumulative  average  cost  for  prime  power  hardware. 
Finally,  the  above  total  was  multiplied  by  one  hundred 
five  percent  to  account  for  the  additional  cost  of 
technical  data.  The  result  was  the  cumulative  average 
radar  production  cost. 

Engineering  development  costs  were  calculated  by 
multiplying   the   cumulative   average   radar  production 
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costs  by  a  factor  of  three  to  get  the  cumulative 
average  cost  for  development  hardware.  He  based  this 
factor  of  three  on  the  historical  trends  of  the  seven 
systems  studied  and  consultation  with  industry  experts. 
The  learning  curve  factor  was  applied  with  the  number 
of  units  to  obtain  the  cumulative  total  development 
hardware  cost.  An  additional  fifteen  percent  was  added 
on  to  account  for  the  extra  cost  of  documentation. 

Dodson's  model  also  provided  a  hierarchy  of 
estimating  systems,  depending  on  the  data  available  to 
the  analyst.  He  recommended  all  information  sources  be 
exhausted  before  reliance  on  estimates  based  solely  on 
the  regression  equations. 

The  next  section  provides  a  brief  overview  of  the 
laser  system  cost  model  developed  by  Dodson  a  decade 
later . 

C.   HIGH-ENERGY  LASER  SYSTEMS  COST  MODEL 

In  June  1979,  Dr.  E.N.  Dodson  completed  a  study  for 
the  U.S.  Air  Force  to  develop  a  comprehensive  life- 
cycle  cost  model  for  high-energy  laser  weapons  systems. 
[Ref.  24]  The  study  accomplished  the  following 
objectives: 

1.  Created  a  cost  data  base  for  laser  weapons 
systems . 

2.  Developed  cost  estimating  relationships  for  laser 
weapons  systems. 

3.  Integrated  the  cost  estimating  relationships  into 
a  life-cycle  cost  model. 

4.  Used  the  cost  model  to  project  costs  for  a  number 
of  weapon  system  concepts. 

Laser  weapons  systems  were  required  to  detect  a 
target  and  establish  its  position.  This  knowledge 
enabled  the   high-energy   laser   beams   to   be  properly 
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directed.  Other  system  requirements  were  target 
identification,  threat  assessment,  firing  doctrine, 
kill  assessment,  plus  miscellaneous  support  equipment. 

Dodson  relied  on  the  parametric  method  to  develop 
cost-estimating  relationships  for  the  laser  systems. 
Since  laser  systems  had  never  been  produced  previously, 
he  used  two  different  approaches  to  gather  the 
necessary  data.  First,  he  used  analogous  equipment  of 
similar  physical  complexity  for  analysis.  For  example, 
1 iquid-propellant  rocket  motors  were  considered  similar 
to  a  laser  device's  mechanical  elements.  Also,  Dodson 
gathered  cost  estimates  for  laser  subsystems  by 
currently  active  contractors  in  the  field. 

Additional  modeling  considerations  included  an 
assessment  of  mission  performance.  Performance  was 
considered  a  system  parameter  measured  at  the 
aggregated  level  of  equipment  detail.  He  developed 
performance-cost  relationships  for  particular  types  of 
designs  at  the  aggregate  level.  Dodson 's  model  was 
capable  of  updating  the  performance-cost  relationships 
with  changes  from  new  technological  developments. 
Examples  of  performance  criteria  for  lasers  were  radar 
detection  range,  laser  power,  mission  kill  probability, 
and  energy  density  on  target. 

Dodson's  next  consideration  in  model  development 
was  the  synthesis  of  the  individual  cost  estimating 
relationships  into  the  overall  cost  model.  He 
recognized  they  must  cover  all  costs  of  interest. 
Also,  costs  were  defined  so  that  double-counting  was 
avoided.  He  also  defined  all  units  in  dollars  of  the 
same  purchasing  power. 

Dodson  based  the  Work  Breakdown  Structure  for  this 
study  on  the  life-cycle  elements  of  the  Demonstration 
and  Validation,  Full  Scale  Development,  Production,  and 
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Operations  and  Support  acquisition  phases.  The  first 
two  phases  separated  recurring  and  nonrecurring  costs. 
Recurring  costs  represented  those  costs  directly 
associated  with  the  fabrication  of  prototypes. 
Nonrecurring  costs  included  all  other  costs,  primarily 
engineering  design  and  component  testing  costs. 

All  cost  relationships  were  presented  in  thousands 
of  1976  dollars.  Provisions  for  translating  those  cost 
figures  into  specified  future  year  constant  dollars 
were  included. 

The  cost  elements  at  the  bottom  levels  of  the  Work 
Breakdown  Structure  included  the  hardware  associated 
with  the  laser  systems  with  other  cost  elements  such  as 
system  test  and  evaluation  and  project  management. 

Computer  subroutines  developed  by  Dodson  enabled 
the  model  to  carry  out  types  of  calculations  common  to 
a  number  of  high-energy  laser  subsystems.  The  required 
inputs  to  the  high-energy  laser  systems  cost  model 
included : 

1.  Choice  of  laser  type,  like  chemical  or  gas 
dynamic . 

2.  Device  output  power  in  watts. 

3.  Specific  power  in  kilojoules  per  pound. 

4.  Number  of  laser  shots  per  mission  load. 

5.  Shot  time  in  seconds  per  pulse. 

6.  Pulse  recurrence  frequency  in  pulses  per  second. 

7.  Electrical  efficiency,  consisting  of  device 
output  power  divided  by  prime  electric  power. 

8.  Output  beam  diameter  in  centimeters. 

9.  Number  of  turrets  per  aircraft. 

10.  Number  of  adaptive  optics  actuators  per  mirror. 

11.  Number  of  flight  hours. 

The   model   also   provided   optional   inputs  to  the 
user.   The  user  could  choose  among  different  procedures 
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for  calculating  fluids  costs,  avionics  costs,  or 
operations  and  support  costs.  For  example,  for  fluids 
the  user  provided  the  flow  rate  for  each  fluid,  which 
was  then  combined  with  cost  and  lasing  time  information 
to  establish  total  fluid  costs.  If  flow  rates  were  not 
available,  the  model  could  assimilate  the  values  of 
specific  power  and  lasing  time  with  internally-stored 
information  about  the  proportions  of  individual  fluids 
and  their  costs,  to  determine  total  fluid  costs. 
Tables  with  the  numerous  characteristics  for  fluids 
were  provided  in  the  model.  Fluids  represented  a 
variety  of  individual  reactants  and  diluents  used  in 
high  energy  laser  systems.  The  primary  characteristics 
contained  were  cost-per-pound  figures. 

The  High-Energy  Laser  Systems  Cost  Model  provided 
the  following  outputs: 

1.  Listing  of  user  provided  system  inputs. 

2.  List  of  Avionics  inputs  as  selected  by  the  user. 

3.  Intermediate  set  of  results  for  fluids,  including 
calculated  values  of  weights  and  costs  for 
individual  fluids. 

4.  Printout  of  fluid-usage  parameters  for  the 
individual  fluids  selected  for  the  case  under 
study. 

5.  Listing  of  the  Operating  and  Support  cost 
parameters  used  in  calculating  these  costs. 

6.  Detailed  presentation  of  life-cycle  costs  by  cost 
element  in  the  Work  Breakdown  Structure. 

Costs  were  presented  in  thousands  of  constant- 
dollars  based  on  1976  as  the  base  year.  Successive 
levels  of  aggregation  were  shown  in  the  cost  model.  The 
model  also  summarized  by  cost  element  the  individual 
cost  estimating  equations  utilized. 
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The  following  example  highlights  Dodson's  model. 
The  validation  phase,  one  of  the  four  life-cycle 
phases,  consisted  of  nonrecurring  and  recurring  costs. 
Nonrecurring  costs  were  the  sum  of  these  elements: 

1.  Airborne  system  costs,  which  included: 

a.  Device  costs. 

b.  Power  supply  costs. 

c.  Fluid  supply  costs. 

d.  Optics,  pointing,  and  tracking  costs. 

e.  Avionics  costs. 

2.  Aircraft  modifications  costs. 

3.  System  test  and  evaluation  costs. 

4.  Project  management  costs. 

The  device  cost  for  airborne  systems  was  the  sum  of 
individual  costs  developed  for  the  device, 
dif fuser /ej ector ,  and  associated  instrumentation 
controls,  along  with  auxiliary  power  elements.  The 
mathematical  cost  estimating  relationship  generated  for 
chemical  laser  devices  was  cost  equalled  15.309  times 
device  output  power  (watts).  Device  output  power  was 
first  scaled  to  the  .44  power. 

Dodson's  model  did  not  express  separately  SOA 
extension  costs.  In  the  next  section,  the  Air  Force 
Spacecraft  model  is  discussed,  with  particular  emphasis 
on  the  use  of  engineering  design  complexity. 

D.   UNMANNED  SPACECRAFT  COST  MODEL 

The  1981  edition  of  the  Space  Division  Unmanned 
Spacecraft  Cost  Model  is  considered  by  the  aerospace 
industry  estimating  community  as  the  most  widely 
applied  spacecraft  cost  estimating  tool.  [Ref.  25] 
The  model's  purpose  is  to  collect  historical 
cost  data  for  use  in  a  parametric  cost  estimating 
relationship  framework.   The  model  is  used  to  formulate 
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more  responsive  cost  estimates  for  long   range  planning 
studies  and  future  spacecraft  systems. 

The  model  is  organized  into  a  data  matrix.  The 
matrix  is  subdivided  into  areas  of  activity, 
subdivisions  of  work,  and  elements  of  cost.  Areas  of 
activity  are  specific  or  general  hardware 
classifications,  such  as  aerospace  ground  equipment. 
Subdivisions  of  work  indicate  processes  associated  with 
more  than  one  hardware  item,  such  as  design  or 
fabrication.  Elements  of  cost  represent  the  standard 
cost  accounting  categories  of  engineering  labor, 
material  costs,  manufacturing  overhead,  and  general  and 
administrative  expenses. 

The  subdivisions  of  work  are  identified  as 
nonrecurring  and  recurring  costs.  A  time-phased  method 
determines  the  break  between  the  two  types  of  effort. 
Nonrecurring  costs  begin  with  concept  development  and 
stop  when  the  qualification  test  of  the  prototype  is 
complete . 

Three  examples  of  the  areas  of  activity  are  the 
platform,  electrical  power  supply, and  the  attitude 
control.  The  platform  is  part  of  the  spacecraft 
structure,  and  bears  the  majority  of  spacecraft  dynamic 
stress  loads.  Examples  of  cost  drivers  for  the 
platform  are:  structure  weight,  volume,  and  mass 
density.  The  electrical  power  supply  generates, 
converts,  and  distributes  all  electrical  energy  between 
spacecraft  components.  Examples  of  cost  drivers  are 
electrical  power  supply  weight,  battery  weight,  total 
vacuum  impulse,  and  action  time. 

The  attitude  control  system  maintains  the 
spacecraft  in  the  required  orbit.  The  system  can  be 
stratified  into  three  design  categories  or  two 
functional  categories  of  equipment.    Cost   drivers  are 
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dry   weight,   wet   weight   (with   fuel),  total  impulse, 
operating  life,  angular  drift,  and  altitude. 
The  ground  rules  for  the  cost  model  are: 

1.  The  model  addresses  only  unmanned  earth-orbiting 
spacecraft . 

2.  Cost  estimating  relationships  are  obtained  by 
relating  costs  at  the  subsystem  level  to  physical 
and  performance  characteristics. 

3.  All  cost  estimating  relationships  are  based  upon 
burdened  costs,   so  the  model  consists  of  the 
total  cost  through  general  and  administrative 
expense  cost  estimating  relationships. 

4.  A  ninety-five  percent  average  learning  curve  is 
used  to  derive  unit  costs. 

5.  The  cost  estimate  is  expressed  in  1979  constant 
dollars . 

Based  on  the  ground  rules,  starting  point  cost 
estimating  relationships  are  generated.  For  the  three 
areas  of  activities  considered  earlier,  the  derived 
regression  equations  were: 

1.  Platform  costs  =  7414.46  +  22.6  X,  with  X 
representing  platform  dry  weight  in  pounds. 

2.  Electrical  Power  Supply  costs  =  360.97  +  .0165  X, 
with  X  representing  the  product  of  electrical 
power  supply   weight  and  beginning  of  life  power 
in  watts. 

3.  Attitude  Control  costs  for  the  attitude  and 
reaction  control  subsystem  =  426.49  +  31.47  X, 
with  X  representing  the  dry  weight   of  the 
attitude  and  reaction  control. 

The  cost  estimating  relationships  listed  above  were 
derived  after  examining  all  program  cost  data  on 
scatter  diagrams.  Next,  regression  analysis  was 
performed  for  several  parameters.   Further  analysis  was 
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performed  on  the  cost  drivers  which  significantly- 
influenced  cost.  Transformations  were  performed  on 
selected  variables  via  multiplication,  square  roots,  or 
logarithms  to  create  synthetic  variables  in  an  effort 
to  find  the  most  influential  cost  drivers.  Finally, 
the  data  was  stratified  for  all  data  points  to 
determine  homogeneity  of  the  points. 

Once  the  cost  estimating  relationships  were 
generated,  they  were  normalized  to  account  for 
inflation,  influences  of  alternate  design  concepts  and 
new  technological  breakthroughs.  In  the  normalization 
process,  actual  cost  data  are  evaluated  with  respect  to 
quantifiable  subjective  parameters.  The  parameters 
enable  the  actual  cost  data  to  be  adjusted  to  a  common 
base  at  the  subsystem  level.  Two  subjective  parameters 
selected  were  technology  carryover  and  complexity  of 
design . 

The  technology  carryover  cost  factor  measures  the 
state-of-the-art  of  technology  at  different  periods  of 
time.  The  technology  carryover  measurement  scale  is 
divided  into  five  levels  to  capture  the  degree  of 
engineering  learning  over  time.   The  five  levels  are: 

1.  1.00:  the  item  is  substantially  beyond  the  SOA. 

2.  .75:  the  item  is  slightly  beyond  the  SOA. 

3.  .50:   the  item   is  within   the  SOA   but  no 
commercial  counterpart  exists. 

4.  .25:   the  item  will  involve  a  minor  modification 
of  commercial  items. 

5.  .10:   the  item  will  be  procured  off-the-shelf. 
Programs  can  be  examined  by  the  Unmanned  Spacecraft 

Cost  Model  by  the  complexity  of  design  cost  factor 
also.  The  first  step  is  to  identify  subsystem 
operational  criteria  which  could  relate  cost  to  the 
degree  of   complexity.    Descriptors  must   be  chosen  to 
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realistically  assess  the  operational  criteria.  Each 
operational  criterion  is  ranked  against  a  base  value  of 
100  percent.  The  evaluation  is  compared  to  the 
subsystem  complexity  factor  matrix.  The  relative 
ranking  indicates  the  degree  to  which  the  operational 
criteria  affects  the  costs  of  developing  the  subsystem. 

To  obtain  the  normalization  cost  factors,  a 
comprehensive  study  of  industry  experts  was  conducted. 
A  weighting  scheme  was  devised  to  generate  one 
normalization  factor  for  each  subsystem  from  the 
composite  of  technology  carryover,  complexity  of 
design,  and  inflation  factors.  Each  subsystem's  raw 
cost  data  points  from  the  initial  cost  estimating 
relationships  were  divided  by  the  composite 
normalization  factor  to  yield  a  set  of  normalized  cost 
data  points.  The  normalized  cost  will  always  be  less 
than  the  initial  point  design  cost  estimate. 

The  normalized  cost  estimating  relationships 
enable  the  cost  analyst  to  perform  trade-off  studies 
for  near-term  conceptual  programs.  They  permit 
calculation  of  costs  for  more  specific  spacecraft 
programs.  The  Unmanned  Spacecraft  Cost  Model  via  an 
appendix  provides  a  summary  of  all  the  normalized  cost 
estimating  relationships. 

The  next  section  outlines  the  basic  concepts  behind 
the  Freiman  Analysis  of  Systems  Technique  (FAST), 
developed  by  Frank  Freiman.  [Ref.26] 

E.   FREIMAN  ANALYSIS  OF  SYSTEMS  TECHNIQUE  (FAST) 

Freiman  developed  the  FAST  parametric  cost 
estimating  system  to  evaluate  the  cost  impact  of 
variations  in  schedule  or  design.  Freiman's  system 
differs  from   the  conventional   parametric  models,  such 
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as  the  Unmanned  Spacecraft  Cost  Model,  in  the  following 
ways : 

1.  It  quantifies  technological  phenomena  underlying 
design  which  cause  costs  to  vary  with  size  and 
design . 

2.  It  reduces  need  for  lengthy  historical  design 
versus  cost  records. 

3.  It  allows  synthesized  data  points  to  be  used  for 
initial  development  estimates. 

The  FAST  methodology  involves  analysis  of  the 
fundamental  concepts  behind  technology  variance  with 
cost  per  pound.  Freiman  found  advanced  SOA 
technologies  provide  more  energy  per  unit  of  design 
mass  than  those  within  the  SOA.  The  same  performance 
for  SOA  extensions  can  be  accomplished  with  less 
equipment  mass.  Freiman' s  theory  is  that  to  advance  a 
design  mass,  more  energy  per  pound  must  be  utilized. 

The  FAST  model  estimates  costs  by  its  class  of 
technology.  Its  seven  basic  types,  in  hierarchial 
order,  are  electronic,  electrical,  heat,  motion, 
mechanical  control,  containment,  and  support.  Freiman 
also  used  the  weighted  average  level  of  the  technology 
with  the  degree  of  performance  desired  to  categorize 
equipment  types  by  design  mass  components.  His  five 
basic  types  of  design  masses  are: 

1.  Energy  Conversion  Mass  -  converts  one  energy  form 
to  another. 

2.  Design  Overhead  Mass  -  added  due  to 
inefficiencies  of  design. 

3.  Application  Mass  -  required  to  transfer  energy. 

4.  Dimensional  Mass  -  required  for  physical  coverage 
of  a  system. 

5.  Conditional  Mass  -  required  for  environmental  or 
personnel  safety  reasons. 
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Based  upon  the  design  mass  and  technology  type, 
FAST  can  quantify  cost  per  pound  of  the  electronic 
system  being  estimated.  Via  computerized  mathematical 
equations  the  cost  estimating  relationships  among  cost, 
weight,  and  technological  complexity  can  be  expressed. 

FAST  also  simulates  the  thought  processes  of 
successful  managers.  FAST ' s  methodology  is  derived 
from  the  way  managers  intuitively  assess  the  cost  of 
SOA  advancements.  FAST  is  designed  to  simulate 
behavioral  responses  ranging  from  establishing  the  data 
base  to  exercising  "what-if"  capabilities. 

FAST  systems  feature  the  following: 

1.  Accept  tailored  inputs  from  varied  design  and 
manufacturing  circumstances. 

2.  Project  funding  requirements  via  graphic  and 
alpha-numeric  displays. 

3.  Provide  risk  measurement  through  display  of  cost 
uncertainties  for  each  cost  segment. 

4.  List  detailed  complexity  values  for  commercial 
and  industrial  items. 

5.  Are  user-friendly,  unlike  the  more  specific  cost 
models  such  as  the  ground  surveillance  radar  cost 
model . 

FAST  is  useful  as  a  check  for  conceptual  stage  cost 
estimates,  although  it  must  be  calibrated  for  each 
individual  user.  Tables  1  and  2  represent  sample 
outputs  for  the  FAST  cost  estimating  model  for 
electronic  equipment.  Table  1  is  divided  into  five 
primary  sections,  broken  down  as  follows: 

1.  Total  estimated  costs  of  the  line  item,  with 
subtotals  for  engineering,  production,  and 
installation  costs.   If  necessary,  costs  for 
schedule  delay  could  be  included. 

2.  Cost  uncertainty  distributions,  for  risk 


57 


evaluation.   Three  confidence  intervals,  ranging 
from  seventy  to  ninety  percent  are  shown  for  the 
engineering,  production,  and  installation 
subtotals.   Note  the  total  assumes  the  covariance 
equals  one. 

Characteristics  of  the  equipment's  energy 
sources.   The  primary  energy  required  is 
expressed  in   kilowatts,  while  secondary  items 
such  as  pressure  and  temperature  are   also 
summarized.    A  synthetic  FAST  energy  variable  is 
calculated  and  shown  below  the  primary  and 
secondary  characteristics. 

Production  cost  data,  using  average  unit  costs. 
Figures  for  the  total  estimated  production  costs, 
manufacturing  costs,  and  the  theoretical  forecast 
are  shown.   In  this  example  manufacturing  costs 
are  the  same  as  production  costs,  due  to  lack  of 
beginning  work  in  process  inventory. 
Input  data.   The  factors  actually  entered  by  the 
cost  estimator  are  presented.   Among  the  inputs 
are  : 

a.  PRJGLOB:   Overall  project  global  inputs  are 
shown.   This  line  requires  an  escalation 
control  factor  for  inflation,  the  year  of 
economics  to  be  used,  learning  curve  factor, 
and  a  cost  multiplier  index. 

b.  FILE  and  FORMAT  input  different  types  of 
system  options. 

c.  GLOB  -  Further  global  inputs  are  included, 
specifically  the  platform  specification 
level,  engineering  design  mass  type,  year  of 
technology  to  be  used  in  production,  and  a 
weighted  energy  value  based  on  kilowatts, 
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british  thermal   units,  pressure,  and 
temperature . 

d.  WTVOL:   Weight  and  volume  factors,  along  with 
their  weighted  values,  are  shown. 

e.  MXLINE:   Manufacturing  complexity  values  for 
the  production  of  the  line  item  are  given. 

f.  PCOST,  ECOST,  ICOST  :   Inputs  for  basic  costs 
of  production,  engineering  and  installation, 
along  with  quantities  and  the  year  of 
economics,  are  shown. 

g.  PSCHD,  ESCHD,  ISCHD  :  Values  for  start  and 
completion  dates,  complexity  values,  and 
skill  levels  are  presented  for  production, 
engineering,  and  installation  schedule 
factors . 

h.   PLOH,  ELOH,  ILOH  :   Production,  engineering, 
and  installation  material,  manufacturing 
overhead,  labor,  and  indirect  labor  factors, 
along  with  an  aggregate  labor  rate  for  each 
phase,  is  presented. 
Table  2  shows  the  cost  distribution   by  total  labor 
hours   for   engineering,   production,  and  installation. 
Also,   a    production    profile    graph    compares   the 
cumulative    funding    for    the   project   against   the 
estimated   cumulative   expenditures,   by   quarters  from 
1984   to    1991.     The   expenditures   indicate   actual 
disbursement   of   manufacturing   costs,   while   funding 
indicate  the  budgeted  figures. 

F.   SUMMARY 

This  section  highlighted  features  of  four  different 

cost  estimation  models.   Chapter  Four  presents  the  case 

study  description   of  Litton   Applied  Technology's  cost 
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estimating   process   for   the   AN/ALR-67   Radar  Warning 
System. 
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LOCATION': 

ITEM: 


TEST  WIT 

GERMANTOWN.MD 

DETAIL 


TABLE    1 

••    TAST-E    • 


DATE:    MON,     JUL    14    1986       09:59    2  3 
FILENAME:     TASTE . EXAMPLE 

ECONFILE: 


DETAIL    ITEM 


SUNITS      -            1,000  BASIC    COST  SCHD    PENALTY 

ENGINEERING  797.19  o.OO 

PRODUCTION  1,301.45  0.00 

INSTALLATION  519.84  0.00 

TOTAL  ACQ.  2,618.48  0.00 


TOTAL  COST 
797.19 

1,301.45 
519 . 84 

2,618. 48 


•  ••••  COST  UNCERTAINTY  DISTRIBUTION 


ENG 

PRCD 
INST 

TOT 


•FROM- 
723 

1,227 

476 

2,427 


70* 


-TO- 
871 

1,376 
564 

2,  810 


•FROH- 
680 

1,184 
450 

2,314 


80»  -TO- 

914 

1,419 

590 

2,923 


-FROM- 
619 

1,  123 
413 

2,155 


90* 


-TO- 
97« 

1,480 
626 

3  ,082 


CHARACTERISTICS  •••** 


PRIMARY 

kilcwat: 


VARIABLE 

(PKW     ) 


SECONDARY 

BRITISH  TH.  UNIT  (SBTU   ) 

PRESSURE  (PSI)  (SPRESS) 

TEMPERATURE  (F)  (STEMP  ) 

FASTE  EQUIVALENT 
FASTE 

TENVLE; FASTE) 


QUANTITY 

250.000 


1, 500.000 

50. COO 

125. 000 


957  .  136< 
FAS  . C . 2^8< 


PRODUCTION  COST  DATA  ••••« 


average  unit  costs  based  on: 
:al  production  cost  manufacturing  cost 

26,029.03  26,029.03 


THEORETICAL  FPCOS: 
40, 466 . 78 


DATA  ••••• 

ESCAL 

GECON 

FPIR 

LCURVE 

rvr  ■•" 

7 

PRJGLCB 

1.  000 

1989 

OCT 

0.  900 

^  n  ^ 

TYPE 

SYSTEM 

RERUN 

REM BOX 

r  ;  -_t 

D 

0 

0 

0 

ALL     COS' 

I    UNCERT 

LOHMAT 

INPUT     FG 

RAFH    FDE 

T 

FORMAT 

1 

0 

0 

0 

0 

0        0 

PLTFM 

EN" 

:ype  matval 

TYEAR 

TEN".' A 

j_ 

GLOB 

1.300 

70 

.000    1.00 

1988 

K\i .  BTU . 

prs.tz::.  :. 

2c: 

WT 

WTFAC 

VOL 

VOLFAC 

WTVOL 

1, 060.70* 

350. CO 

24  . 

15*      100 

.00 

PMX 

MXTYPE 

ELWT 

ELMX 

STMT 

STXX 

HXLINE 

0.200 

4.280* 

0.0 

0.000 

0.0 

0.03 

PCOST 

PQTY 

PECON 

FOAK 

PCOST 

1, 301.45* 

50. 

1989 

1  .  000 

PSTART 

PFIN 

PSCHDX 

PRDMX 

PYEARC 

PDISC 

PSCHO 

DEC. 1989 

NOV. 1991* 

100.00 

3.  327* 

1989* 

NONE 

PMATL 

PMOH 

PLABOR 

PLOH 

PLRATE 

PLOH 

0.30 

0.  00 

0.34* 

1.  06* 

11.75 

ECOST 

EQTY 

EECCN 

EMX 

ENEW 

ECOST 

797.  19* 

5  .  2 

1989 

0.  850 

0.  700 

ESTART 

EFIN 

ESCHDX 

ELEVEL 

EYEARC 

EDISC 

ESCHD 

JUN. 198  8 

OCT. 1989 

121.43* 

0.  700 

1989* 

NONE 

EMAIL 

EMCH 

ELABOR 

ELOH 

ELRATE 

ELCH 

0.35 

0.  00 

0.26* 

1.48* 

22.55 

ICOST 

IQTY 

IECON 

IMX 

ICOST 

519.84* 

50. 

1989 

1.250 

ISTART 

IFIN 

ISCHDX 

ILEVEL 

IYEARC 

IDISC 

ISCHD 

JUL. 1990 

FEB. 1991* 

80.  00 

1.  000 

1989* 

NONE 

IMATL 

IMOH 

ILABOR 

ILOH 

ILRATE 

ILOH 

0.15 
CONTIN 

0.  00 

ADD 

0.46* 

0.8S* 

13.  40 

END 

0 

0 
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LABOR 

(HRS) 

COST 

OVHD 

ST 

MATL 

OVHD 

ST 

TOT 


ENGINEERING 

9,283.58)      ( 

209.34 
308 . 83 

518. 18 

279.02 

0.  00 

279.02 

797.19 


TABLE    2 
COST    DISTRIBUTION    •  •••• 
PRODUCTION  INSTALLATION 

37,572.70)     (  17,868.99)     ( 


441.48 
469.54 

911.02 

390.  44 

0.  00 

390.44 

1,301.45 


239.44 
202.42 
441.86 

77.98 

0.00 

77.98 

519.84 


TOTAL 

64,725.27) 

890. 27 

980.79 

1,871.06 

747.43 

0.  00 

747.43 

2,618.41 


'PROJECT:       TEST   UNIT 
LOCATION:    GERMANTOWN, MD 
ITEM:  DETAIL 


FAST-E 


DATE:  HON,  JUL  14  198  6   09:59:51 
FILENAME:  FASTE . EXAMPLE 

ECONFILE: 


DETAIL  ITEM  ***** 


T 
H 
0 

u 

S 
A 

N 

D 
S 

• 

1 
3 
3 


89 


90 


15  -b  •  basic  exp, 
'  *  -  est.  fund 


PRODUCTION  PROFILE 
91  92 


89 


bbbbbbb 


90  91  92 

YEARS 

(1  MONTH  PER  INCREMENT 


93 


93 


94 


94 


o2 


TABLE  2  (COMT. ) 

••••*  ESTIMATED  PROJECT  PROFILE  ***** 
(COST  IN  THOUSANDS) 

MONTH  YEAR   CUM.  FUNDINGS   CUM.  EXPENDITURES 

OCT    1989          316.3  0.0 

JAN    1990          816.3  66.4 

APR    1990           816.3  321.2 

JUL    1990           816.3  628.5 

OCT    1990         1,295.3  899.2 

JAN    1991        1,295.3  1,09  5.5 

APR    1991        1,295.3  1,215.4 

JUL    1991        1,295.3  1,276.5 

OCT    1991         1,301.5  1,299.9 

NOV    1991         1,301.5  1,301.5 


ESTIMATED  FUNDINGS 
(COST  IN  THOUSANDS) 


MONTH  YEAR  FUNDINGS 

OCT  1989  816.3 

OCT  1990  479.1 

OCT  1991  6.1 

PRODUCTION  QUANTITY  BY  FISCAL  YEAR 

YEAR  ENDING 

MONTH   YEAR  FISCAL  QTY 

SEP     1990  1 

SEP     1991  48 

SEP     1992  1 

TOTAL  50 
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IV.  DESCRIPTION  OF  LITTON  APPLIED  TECHNOLOGY'S  COST 

ESTIMATING_SYSTEM 


A.  INTRODUCTION 

This  chapter  describes  Litton  Applied  Technology's 
cost  estimating  system  for  development  projects  which 
extend  technology  beyond  the  SOA .  This  case  study  will 
primarily  focus  on  the  AN/ALR-67  Radar  Warning 
Receiver,  also  called  the  Advanced  Countermeasures 
Control  and  Warning  System.  The  chapter  is  divided 
into  the  following  sections: 

1.  Litton 's  background  in  development  of  radar 
warning  receivers. 

2.  Description  of  key  physical  and  performance 
characteristics  of  the  AN/ALR-67  Radar  Warning 
Receiver . 

3.  Litton's  pricing  and  estimation  system. 

4.  Cost  estimation  methodology  for  derivation  of 
direct  costs. 

5.  Delineation  of  actual  costs  incurred  by  Litton 
from  development  through  initial  production  for 
the  AN/ALR-67  Radar  Warning  Receiver. 

B.  BACKGROUND  FOR  RADAR  WARNING  RECEIVERS 
1 .   General  Background 

Litton  Applied  Technology  is  a  division  of 
Litton's  Electronic  Warfare  Systems  Group.  Currently 
located  in  south  San  Jose,  California,  Litton  Applied 
Technology  employs  approximately  1800  people  in  the 
research,  design,  manufacture,  and  support  of  defense 
electronics  systems.  Besides  radar  warning  receivers, 
they  also  specialize  in  integrated  electronic  warfare 
systems,  space  and  strategic  defense  systems,  and 
flight   training   and   operational   simulation  systems. 
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Applied  Technology  actually  was   an  independent  company 
until  1983,  when  it  became  a  division  of  Litton. 
2 .   History  of  Radar  Warning  Receivers 

Applied  Technology  first  became  involved  with 
radar  warning  receivers  in  1965.  The  concept  of  radar 
warning  receivers  was  initiated  by  a  special  Air  Force 
task  force  commissioned  to  develop  methods  to  counter 
Soviet  surface-to-air  missiles.  In  1965,  North  Vietnam 
developed  a  complex  air  defense  system  closely 
coordinated  through  the  use  of  communications  and 
radar,  based  on  Soviet  technology.  The  Air  Force's 
research  committee  reviewed  new  concepts  for  warning 
and  jamming  equipment  which  could  counter  the  North 
Vietnamese  air  defense  threat.  In  November  1965, 
Applied  Technology  received  a  six  million  dollar 
production  contract  for  500  AN/APR-25  Radar  Warning 
Receivers  from  the  USAF  Sacramento  Air  Material  Area. 
The  contract  was  Applied  Technology's  first  major 
production  contract,  since  previous  defense  experience 
was  limited  to  technology  applications  for 
intelligence  programs,  where  large  production  runs 
consisted  of  only  ten  units. 

The  design  philosophy  for  the  AN/APR-25 
centered  on  gathering  as  much  signal  information  as 
possible.  Crystal-video  detection  techniques  were  used 
in  the  threat  bands  to  determine  the  hostile 
equipment's  relative  direction. 

In  the  late  1960 's  Applied  Technology  updated 
the  AN/APR-25  with  the  AN/APR-35  Radar  Warning 
Receiver.  New  technology  such  as  automatic  time/video 
correlation  circuits  and  a  new  superheterodyne  analysis 
receiver  were  added.  Also,  improvements  in  operator 
interfaces  were  added  to  the  equipment  to  enable  the 
Electronic     Warfare     Officer    to    instantaneously 
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communicate  to   the  pilot   which  directions  surface-to- 
air  missiles  were  fired  from. 

The  next  generation  of  radar  warning  receivers 
was  introduced  in  1970,  based  on  the  new  Soviet  threat 
to  the  Mediterranean  region.  The  AN/ALR-45  became  the 
first  digital  system  to  incorporate  hybrid 
microcircui ts  using  digital  logic  and  clock  drivers. 
This  generation's  design  philosophy  emphasized  the  need 
for  sorting  analysis  of  emitter  types  and 
prioritization  of  lethal  pulses.  Non-lethal  threat 
information  could  now  be  discarded. 

In  1971  Applied  Technology,  in  a  fundamental 
change,  became  deeply  involved  in  computer  design 
evolving  from  analog  circuit  design.  In  1972  the 
company  developed  the  Applied  Technology  Advanced 
Computer  (ATAC)  specifically  for  electronic  warfare. 
This  computer  was  capable  of  being  reprogrammed  at  a 
squadron  level.  The  ATAC  computer's  volume  was  96 
cubic  inches,  with  power  consumption  45  watts,  and  an 
input/output  rate  of  1.25  megawords  per  second. 

In  1975,  the  ATAC  computer  was  used  in  the 
development  of  the  AN/ALR-67  threat  warning  program. 
Integrated  power  management  systems  now  collect  and 
analyze  multiple  threats  to  enable  optimum  jamming. 
The  AN/ALR-67  was  developed  due  to  the  Navy's  need  for 
increased  speed  and  prioritized  threat  warning 
information.  The  AN/ALR-67  is  now  deployed  on  the  F/A- 
18A,  CF-18,  F-14A,  F-14B,  A-6E,  A-6F,  AND  AV-8B 
aircraft . 

The  next  generation  of  radar  warning  receivers 
to  be  fully  deployed  will  be  the  AN/ALR-74  threat 
warning  systems,  as  of  1987.  The  AN/ALR-67  is 
completing  the  full-scale  development  phase. 
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C.   PHYSICAL  AND  PERFORMANCE  CHARACTERISTICS 

The  AN/ALR-67  Radar  Warning  Receiver's  primary- 
function  is  to  notify  the  radar  operator  of  the 
presence  of  threatening  signals.  The  AN/ALR-67  is 
designed  to  inform  the  pilot  of  how  many  hostile  radar 
systems  are  active  in  his  flight  area.  The  AN/ALR-67 
identifies  the  type  and  relative  location  of  each 
threatening  radar  signal. 

The  most  important  performance  criterion  of  the 
AN/ALR-67  is  response  speed.  The  AN/ALR-67  is  designed 
to  allow  the  pilot  time  for  evasive  action  against 
potential  threats.  Also,  the  radar  warning  receiver's 
response  time  is  quick  enough  to  provide  accurate 
relative  direction  information  on  hostile  threats 
immediately  following  aircraft  maneuvers. 

To  be  effective,  the  AN/ALR-67  must  transmit  the 
critical  parameters  of  the  threat  to  the  aircraft 
jamming  system  in  digital  form.  The  effectiveness  of 
the  jammer's  electronic  countermeasures  depends  on  the 
AN/ALR-67's  receiver  acquisition  time. 

The  second  performance  criterion  for  the  AN/ALR-67 
is  threat  identification.  The  radar  warning  receiver 
characterizes  hostile  signals  by  their  modulation 
characteristics  and  range  of  RF  frequency  operation. 
RF  frequency  pulse  trains  often  occur  simultaneously, 
so  the  AN/ALR-67  must  be  capable  of  unambiguous  signal 
identification. 

The  AN/ALR-67  is  also  designed  to  look  at  selective 
frequency  bands  so  high  duty  signals  can  be  analyzed 
independently  without  interference.  The  pilot  receives 
information  from  the  AN/ALR-67  Radar  Warning  Receiver 
by  visual  displays  on  a  three-inch  diameter  cathode  ray 
tube  in  the  cockpit.  The  pilot  receives  data  as 
following  from  the  AN/ALR-67. 
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1.  Relative  direction  of  each  signal  to  an  accuracy 
of  15  degrees  to  30  degrees. 

2.  Indication  of  each  signal's  strength. 

3.  Identification  by  symbols  of  specific  radar 
types . 

The  design  of  the  AN/ALR-67  Radar  Warning  Receiver 
is  constrained  by  three  practical  factors:  volume, 
power  consumption,  and  cost.  The  allocated  volume  for 
the  radar  warning  receivers  has  remained  constant  since 
1965.  The  increase  in  customer  requirements  and  the 
continued  development  of  electronics  microwave 
techniques  has  led  to  high  package  density,  which 
imposes  the  constraints  on  power  consumption.  Cost 
constraints  must  be  met  through  system  tradeoffs.  A 
six  decibel  difference  in  receiver  sensitivity 
requirements  could  mean  the  difference  between 
development  of  crystal  video  receivers,  or  more 
expensive  wideband  superheterodyne  receivers. 

Processing  of  signals  through  the  ATAC  computer  has 
replaced  human  interpretation  of  audio  and  visual 
information.  The  AN/ALR-67  has  a  tangential  receiver 
sensitivity  between  -50  dBm  and  -60  dBm,  at  a  10  -MHZ 
video  bandwidth.  System  sensitivity  experiences  losses 
due  to  cabling  and  filtering. 

Search  speed  limitations  place  another  constraint 
on  the  AN/ALR-67.  The  ratio  of  the  RF  frequency  search 
band  to  the  receiver  instantaneous  bandwidth  determines 
the  length  of  time  it  takes  a  receiver  to  intercept  a 
potential  signal. 

The  most  recent  advances  in  microprocessor 
technology  enable  radar  warning  receivers  to  have  each 
element  of  the  distributed  receiver  system  to  be 
controlled  independently  by  a  computer.  The  AN/ALR-67 
was  the  first  radar  warning  receiver  to  control  its 
system  operations  via  computer  software. 
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The  individual  subsystem  components   of  the  AN/ALR- 
67  Radar  Warning  System  are  the: 

1 .  Computer . 

2.  Azimuth  indicator. 

3.  Control  indicator. 

4.  Quadrant  receiver. 

5.  Special  receiver. 

6.  Receiver  antenna. 

7.  Quadrant  antenna. 

Their  size   and  weight  characteristics  are  shown  in 
Table  3. 
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D.   LITTON  APPLIED  TECHNOLOGY'S  COST  ESTIMATING  SYSTEM 

Litton  Applied  Technology  has  developed  a  complex 
matrix  structure  to  manage  its  proposal  estimating 
system  for  government  contracts  such  as  the  AN/ALR-67 
Radar  Warning  Receiver.  Its  approach  is  designed  to 
accommodate  government  procurement  regulations, 
compliance  with  the  Cost  Accounting  Standards,  specific 
customer  requirements,  and  its  own  company  policies. 
The  stated  goal  of  their  cost  estimating  system  is  to 
be  systematic  and  consistent. 

Litton's  Proposal  Pricing  and  Estimating  Manual 
defines  its  five  primary  tasks  as: 

1.  Define  requirements  in  a  manner  which  allows 
specific  work  elements  to  be  performed. 

2.  Develop  a  work  breakdown  structure  and  cost 
matrix  compatible  with  the  cost  collection 
system  of  the  cost  elements.   The  elements  must 
be  measurable  and  definable  through  task 
descriptions  which  are  consistent  with  the 
statement  of  work  requirements. 

3.  Identify  and  develop  significant  milestones  and 
schedules  for  each  work  element  and  a  realistic 
program  schedule. 

4.  Prepare  data  to  serve  as  the  basis  for  the  review 
of  all  cost  estimates. 

5.  Review  and  present  the  related  cost  experience, 
historical  data,  and  detailed  cost  estimates. 

The  Vice  President  of  Business  and  Financial 
Operations  has  overall  responsibility  for  Litton 
Applied  Technology's  cost  estimation  system.  The  two 
primary  subordinates  who  assist  him  are  the  Director  of 
Proposal  Cost  Estimating  and  Analysis  and  the  Manager 
of  Proposal  Operations. 

The  Director  of  Proposal  Cost  Estimating  and 
Analysis  is   primarily  responsible   for  the  development 
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of  cost  proposals  for  division  products.  His  division 
must  establish  and  maintain  effective  cost  estimating 
tools  and  techniques.  He  also  ensures  cost  proposals 
comply  with  customers'  cost  proposal  requirements  and 
regulations.  The  division  must  support  internal  cost 
reviews  and  establish  the  budget  baselines  for  the 
project  cost  control.  The  direct  planning,  monitoring, 
and  prioritization  of  all  cost  proposal  activities  are 
conducted  under  his  cognizance. 

Litton  Applied  Technology's  corporate  guidelines 
hold  the  Director,  Cost  Estimating  and  Analysis 
directly  accountable  for: 

1.  Formulation  of  guidelines  for  all  proposal 
pricing  preparation  and  reviews  in  accordance 
with  division  policy. 

2.  Development  of  overall  division  cost  proposal 
plans.   He  must  define  an  schedule  the 
prerequisite  support  from  line  management.   The 
plans  must  cover  the  statement  of  work,  basic 
assumptions,  cost  data,  and  problem 
identification  and  resolution.   The  conducting  of 
reviews  with  line  management  on  major  cost 
proposals  is  included  within  this  task. 

3.  Support  of  cost  proposals  during  the  customer's 
evaluation  cycle.   He  must  develop  and  establish 
negotiation  cost  positions.   He  also  assigns  and 
approves  the  members  of  the  negotiating  team. 
The  director  ensures  cost  updates  and  cost 
disclosures  are  in  accordance  with  customer 
regulations . 

4.  Review  current  division  cost  performance  trends 
to  ensure  that  such  performance  is  considered  in 
all  proposals. 

5.  Direct  resources  dedicated  to  the  development  and 
maintenance  of  the  pricing  data  base.   He  is  also 
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accountable  for  implementing  advanced  estimating 

techniques  and  systems  for  the  division's 

proposals . 
6.   Support  design-to-cost  analysis  projects  of  the 

division . 
The  Manager  of  Proposal  Operations  is  accountable 
for  managing  Litton  Applied  Technology's  capability  to 
respond  to  customer  requests  for  cost  estimates.  He 
ensures  the  division  cost  proposals  are  complete, 
accurate,  and  minimize  cost  risk.  He  serves  as  a  key 
interface  with  the  functional  line  departments,  program 
office,  senior  management,  auditors,  and  customers. 
Finally,  he  must  review  and  approve  all  formal  cost  and 
price  proposals  for  content  and  conformance  with 
government  regulations,  public  law,  and  customer 
requirements . 

The  cost  proposal  cycle  for  Litton  Applied 
Technology  consists  of  thirteen  distinct  steps.  The 
individual  steps,  in  successive  order,  are: 

1.  Receipt  and  acceptance  of  the  customer's  request 
for  proposal . 

2.  Assign  a  proposal  team  manager. 

3.  Issue  the  proposal  authorization  to  proceed 
order . 

4.  Assemble  the  proposal  team,  with  representatives 
from  Engineering,  Quality  Assurance,  Operations, 
Contract  Administration,  and  Proposal  Cost 
Estimating  and  Analysis. 

5.  Conduct  proposal  team  planning  meetings.   The 
team  must  establish  a  tentative  program  schedule 
and  develop  the  work  breakdown  structure.   The 
bid  matrix,  which  designates  the  individuals 
responsible  for  the  cost  estimate  for  every 
single  element,  is  promulgated. 

6.  Brief  senior  management  of  the  team's  plans  at  a 
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one-hour  kickoff  meeting. 

7.  Review  the  proposal  strategy  and  technical 
approach  at  previously  defined  Intervals. 

8.  Identify  all  material  requirements. 

9.  Collect  and  analyze  the  cost  estimates. 

10.  Conduct  final  management  review. 

11.  Prepare  the  proposal  submittal  volume. 

12.  Submit  the  proposal  to  the  customer. 

13.  Complete  audits  and  negotiations  by  the  customer. 
The  key  member   in   this   matrix   concept   for  cost 

estimating  is  the  proposal  analyst  assigned  from  the 
Proposal  Cost  Estimating  and  Analysis  Division.  He  is 
responsible  for  the  compilation  and  analysis  of  the 
estimated  costs  submitted  by  the  personnel  designated 
in  the  bid-matrix  structure.  Also,  he  performs  "make- 
or-buy-analysis"  on  selected  high  dollar  value  parts. 

The  general  cost  estimating  approach  by  the 
proposal  team  Is  the  "bottoms-up"  cost  engineering 
method.  The  bottoms-up  method  is  utilized  for  every 
proposal,  from  development  of  new  designs  to  full-scale 
production  cost  proposals.  The  key  steps  the  team 
performs  for  cost  analysis  are: 

1.  For  existing  design  configurations,  the  proposal 
analyst  retrieves  a  computerized  bill  of  material 
from  the  on-line  material  pricing  system. 

2.  For  new  designs,  the  responsible  functional 
organization  will  generate  a  bill  of  material  and 
send  it  to  the  proposal  analyst. 

3.  The  proposal  analyst  develops  a  priced  bill  of 
material  with  support  from  the  Procurement 
Material  Pricing  Department  for  inclusion  in  the 
cost  proposal . 

4.  The  functional  organizations  submit  their  direct 
and  indirect  labor  estimates  to  the  proposal 
manager  for  review. 
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5.  All  functional  cost  estimates  are  given  to  the 
proposal  analyst  to  create  the  preliminary  cost 
roll-up . 

6.  Preliminary  cost  reviews  with  senior  management 
are  conducted. 

7.  The  proposal  analyst  receives  the  cost  review 
results.  He  next  compares  them  with  the  cost 
history  on  similar  programs  for  reasonableness. 

8.  All  team  members  review  the  revised  cost 
estimates  to  ensure  they  are  factual,  verifiable, 
complete,  and  support  the  proposed  amounts. 

9.  The  proposal  manager  and  proposal  analyst  brief 
the  company  executives  at  the  Final  Management 
View  to  defend  their  cost  estimates. 

For  every  stage  in  the  cost  proposal  development 
process,  checkoff  sheets  are  generated.  Signatures  by 
the  responsible  individuals  are  obtained  to  indicate 
completion  for  every  step.  The  proposal  analyst 
primarily  verifies  the  functional  groups'  cost 
estimates  for  labor  hours  and  material  quantity.  On 
production  contract  proposals  with  little  risk,  the 
proposal  analyst  generates  the  cost  estimate  himself 
based  strictly  on  historical  data.  Company  officials 
estimate  that  an  average  of  three  complete  cost 
estimate  reviews  are  conducted  before  final  approval. 

E.   COST   ESTIMATION    METHODOLOGY   FOR   DERIVATION   OF 

DIRECT  COSTS 

Litton  Applied  Technology's  Cost  Estimation  and 
Pricing  Manual  defines  direct  material  as  the  cost  of 
material  used  in  making  a  product  which  is  directly 
associated  with  a  change  in  the  product.  Litton's 
direct  material  base  is  comprised  of  raw  materials, 
purchased  parts,  and  subcontracted  items.   For  raw 
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materials,  a  raw  stock  factor  is  applied  to  the  total 
estimated  material  base  to  account  for  these  costs.  In 
their  system  raw  materials  which  require  further 
processing  are  treated  as  indirect  costs. 

When  budgetary  estimates,  also  called  ROM  (rough 
order  of  magnitude)  estimates,  are  requested  for 
supplied  of  existing  products,  they  are  based  on  the 
most  recent  firm  estimates  of  the  same  or  similar 
items.  The  prior  estimate  is  adjusted  by  the  analyst 
for  quantity  differences  and  the  degree  of  complexity. 
An  annual  escalation  factor  is  added  to  the  prior 
material  estimate  to  account  for  period  of  performance 
differences.  The  difference  in  quantity  is  adjusted  by 
utilization  of  a  ninety-five  percent  improvement  curve. 
The  proposal  manager,  assisted  by  the  functional 
engineering  team  members,  provide  the  complexity  factor 
for  material  estimates  to  the  cost  analyst. 

Follow-on  spare  parts  estimating  is  based  on  Litton 
Applied  Technology's  on-line  computerized  material 
pricing  system.  The  source  data  for  the  computerized 
system  represents  the  most  current  configuration  for 
released  part  lists.  Proposal  analysts  have  the 
capability  to  extract  the  purchase  order  history  of 
selected  spare  parts  dating  back  to  1975.  The  analysts 
incorporate  this  information  to  the  priced  bill  of 
material.  The  computerized  on-line  system  can  generate 
a  priced  bill  of  material  by  either  the  individual 
assembly  or  part  number,  or  a  consolidated  group  of 
part  numbers.  The  computer's  primary  files  from  which 
the  bills  of  material  are  generated  are  the  engineering 
configuration  file  and  the  manufacturing  configuration 
file. 

The  Proposal  Material  Audit  Report  is  Litton's  name 
for  the  computer  generated  list  of  material  prices 
based  on  the  purchase   order  history.    Litton's  policy 
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for  high  dollar  value/high  usage  parts  dictates  a 
preference  for  current  vendor  quotes  instead  of  sole 
reliance  on  previous  history.  Their  normal  procedures 
mandate  at  least  three  competitive  quotes  must  be 
obtained.  Follow-on  spare  parts  are  also  adjusted  in 
quantity  by  Litton's  ninety-five  percent  improvement 
curve.  Escalation  factors  for  the  period  of 
performance  differences  also  are  included. 

If  there  is  no  previous  history  of  a  material 
purchase,  the  proposal  analyst  requests  via  a  standard 
material  pricing  request  form  that  the  Purchasing 
Material  Pricing  Department  obtain  vendor  quotes.  The 
functional  proposal  team  members  provide  the  physical 
description  to  the  proposal  analyst. 

Estimates  of  material  requirements  for  the  SOA 
development  projects,  such  as  the  conceptual 
exploration  phase  of  the  AN/ALR-67  Radar  Warning 
System,  are  based  on  the  "bottoms-up"  system 
engineering  approach.  The  cognizant  engineer  at  the 
lowest  level  of  the  work  breakdown  structure  develops 
the  bill  of  materials  after  detailed  analysis  of  the 
proposed  design  configuration  specifications.  The 
responsible  engineer  provides  the  quantity,  part 
number,  and  description  of  the  required  materials  in 
accordance  with  the  bid  task  matrix  instructions.  The 
required  bill  of  material  is  passed  to  the  proposal 
analyst  via  the  Material  Cost  Estimate  Detail  Form. 

The  material  requirements  for  new  developments  are 
based  on: 

1.  Similarity  to  existing  equipment. 

2.  Vendor  catalog  items. 

The  priced  bill  of  material  based  on  "similar  to" 
equipment  is  adjusted  by  a  qualitative  complex  factor. 
For  items  never   previously   purchased,   the  complexity 
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factor  is   determined  by   a  consensus   among  members  of 
the  proposal  team. 

Once  the  dollar  estimates  for  the  material 
estimates  are  gathered,  the  proposal  analyst  next 
develops  the  Principle  Items  List.  This  list  will 
include  a  previously  determined  numerical  sampling  of 
high  dollar/high  usage  items.  The  written  rationale 
for  each  item  must  include: 
1  .   Part  number . 

2.  Known  or  anticipated  source. 

3.  Total  quantity. 

4.  Unit  and  total  price. 

5.  Competition  status. 

6.  Basis  for  establishing  the  source. 

7.  Determination  for  reasonableness  of  cost. 

The  proposal  analyst  justifies  cost  reasonableness 
for  the  selected  items  based  on  inputs  from  the 
Procurement  Material  Pricing  Department.  The  proposal 
analyst  next  presents  the  list  for  approval  to  the 
proposal  and  business  area  managers  during  formal  cost 
input  coordination  meetings. 

The  Direct  Labor  Narrative  Statement  is  another 
document  required  for  the  cost  estimate  of  the 
proposal.  Upon  completion,  both  the  functional 
proposal  team  representative  and  the  cognizant 
functional  director  are  required  to  sign  it.  The 
document  should  show  enough  detail  to  separate  labor 
for  each  distinct  operation.  Each  operation  is 
identified  by  an  engineering  cost  center  and  labor 
category.  The  estimates  for  direct  labor  costs  must 
consider  whether  prior  relatable  efforts  exist  or  if  no 
verifiable  labor  cost  data  can  be  found.  The  estimates 
are  performed  at  the  lowest  level  of  the  work  breakdown 
structure . 
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For  prior  relatable  efforts,  the  copy  of  the 
previous  cost  report  is  attached  if  the  estimated  hours 
bid  are  identical  or  directly  related  in  terms  of 
equivalent  technical  complexity.  The  proposal  cost 
analyst  will  review  the  incurred  labor  hour  costs  on 
sample  task  work  orders  to  audit  the  validity  and 
accuracy  of  the  prior  related  task. 

The  proposal  analyst  must  search  for  more  detailed 
evidence  if  the  prior  relatable  effort  is  of  varying 
complexity  to  the  present  effort.  The  labor  hours 
currently  estimated  are  compared  directly  to  a  similar 
effort  for  which  verification  of  incurred  hours  exist. 
The  proposal  analyst  compares  the  present  effort  to  the 
prior  related  task  by  developing  a  ratio  based  on  the 
relative  technical  complexities  of  the  two  tasks.  The 
percentage  difference  must  be  explained  by  identifiable 
documentary  evidence  such  as  the  following: 

1.  Differences  in  number  of  units  to  be  assembled. 

2.  Variance  in  assembly  component  count. 

3.  Differences  in  number  and  type  of  cables. 

4.  Differences  in  testing  requirements. 

5.  Comparison  of  technical  and  performance 
differences . 

6.  Size  or  weight  differences. 

To  achieve  consistency  on  the  Direct  Labor 
Narrative  Report,  the  prior  effort  is  assigned  a 
complexity  base  value  of  one.  The  narrative  portion 
should  specifically  cite  specification  paragraphs  which 
account  for  the  difference  in  the  plus  or  minus 
technical  requirement. 

If  no  background  verifiable  data  can  be  found,  the 
narrative  should  explain  how  the  direct  labor  costs  are 
derived.  Sources  such  as  conceptual  estimating  guides 
should  be  indicated  by  title;  for  example,  "Electronic 
Cost  Estimating   Data",  by   Fred  Hartmeyer.    The  basis 
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for  level  of  effort  proposal  estimating  should  also  be 
stated.  The  reasons  or  rationale  for  labor  cost 
estimates  which  are  purely  judgmental  is  also 
documented  by  the  proposal  analyst. 

The  cost  accounting  system  for  Litton,  incorporated 
in  the  computerized  on-line  system,  provides 
historical  data  at  the  work  order  level,  task  level, 
and  project  level  by  expense  center  code,  for  direct 
labor  costs.  An  annual  labor  escalation  factor 
consistent  with  that  utilized  in  direct  labor  rate 
projections  is  applied  to  estimates  based  on  prior 
related  efforts. 

The  historical  data  utilized  for  direct  labor 
estimating  includes  the  following: 

1.  Direct  Labor  Hour  Audit  Report:   shows 
manufacturing  work  order  closures  for  the  latest 
two  years  by  assembly  number.   A  ratio  for  unit 
average  hours  by  expense  center  code  for  each 
part  number  is  derived  by  summation  of  the 
manufacturing  work  order  closure  hours. 

2.  Proposal  Direct  Hours  Report:   shows  two  years  of 
labor  hours  data  for  all  subassemblies  in  the 
requested  assembly.   This  report  is  utilized  for 
manufacturing  related  tasks. 

3.  Material  Work  Order  History  Cost  Estimating 
Summary  Report:   summarizes  total  unit  hours  by 
expense  center  code  for  the  requested  assembly 
which  Litton  will  locally  manufacture. 

4.  Contract  Cost  Status  Report:   provides,  for  non- 
manufacturing  tasks,  the  historical  data  base 
for  similar  tasks. 

Litton  Applied  Technology's  methodology  for  direct 
labor  estimates  for  development  programs  is  similar  to 
the  process  described  earlier  for  all  labor  hours 
estimates.    The   only   added   feature   for  development 
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program  estimates  is  an  increased  emphasis  on 
documentation  via  narrative  analysis  for  complexity 
adjustments.  The  proposal  analyst  applies  appropriate 
direct  labor  hour  bid  rates.  The  proposed  bid  rates 
are  determined  by  examining  the  period  of  performance 
of  the  program  plan  and  arriving  at  the  midpoint  of  the 
effort.  Historical  bid  rates  are  determined  by 
dividing  the  quarterly  total  dollars  expended  by  the 
quarterly  total  labor  hours  incurred  for  each  expense 
center  code. 

The  other  direct   costs   included   in   every  Litton 
development  or  production  program  estimate  are: 

1.  Vendor  Nonrecurring  and  Tooling:   These  represent 
the  vendor  costs  associated  with  development, 
start-up,  and  tooling  costs  to  produce  and 
deliver  equipment.   The  basis  for  this  estimate 
consists  of  written  vendor  quotations  or 
historical  data  such  as  prior  purchase  orders  or 
project  cost  reports. 

2.  Travel:   Travel  estimates  consist  of 
transportation  and  subsistence  costs  directly 
associated  with  the  program  estimate. 

3.  Field  Service  Differential:   These  costs  include 
additional  compensation  over  and  above  base 
salary  expenses  and  per  diem,  which  serve  as  an 
incentive  to  field  support  personnel  on 
assignment . 

4.  Service  Centers:   The  three  service  centers  are 
Reprographic  Services,  Programming  Services,  and 
Word  Processing  Services.   The  cost  estimates  use 
hourly  billing  rates  based  on  forecasted 
utilization  and  operational  costs  for  the  service 
center.   The  costs  are  estimated  either  by  prior 
verifiable  experience  or  the  estimated  number  of 
hours  the  service  will  take. 
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Litton  Applied  Technology  also  uses  estimates  of 
direct  labor  hours  in  their  derivation  of  indirect  pool 
costs.   The  six  indirect  pools  are: 

1.  Material  overhead:   This  rate  is  determined  by 
the  ratio  of  annual  total  indirect  costs  for  each 
expense  center  code  divided  by  the  total  dollar 
value  of  the  material  base,  including  scrap  and 
raw  stock. 

2.  Fringe  benefits:   This  rate  is  calculated  by  the 
ratio  of  annual  fringe  benefit  expense  costs 
divided  by  the  total  division  labor  dollars 
expended . 

3.  Sunnyvale  plant  overhead  and  Georgia  operations 
overhead:   Both  pool  rates  are  based  on  total 
plant  overhead  dollars  expended  annually  divided 
by  total  division  direct  labor  dollars  plus  their 
associated  fringe  benefit  costs. 

4.  Field  service  overhead:   This  rate  is  based  on 
total  annual  overhead  costs  divided  by  direct 
labor  costs. 

5.  General  and  administrative  expenses:   These 
expenses  consist  of  independent  research  and 
development,  bid  and  proposal  cost  estimating, 
executive  staff,  general  accounting,  resource 
allocation  and  control,  and  defense  systems 
business  development  costs.   The  rate  is  based  on 
total  expenses  divided  by  the  forecasted  total 
costs  sum  allocable  to  contracts. 

Besides  the  indirect  rates,  direct  factors  such  as 
manufacturing  overtime  premium,  material  raw  stock,  and 
manufacturing  and  engineering  support  services  are  also 
allocated.  Engineering  support  functions  include 
software  engineering  and  development,  and  engineering 
design  support.  Manufacturing  support  services  include 
operations  control,   shipping,  and  the  test  directorate 
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staff.  The  rate  for  these  functions  is  based  on  the 
ratio  of  the  six  month  historical  average  of  direct 
labor  hours  for  each  service. 

F.   ACTUAL   COSTS   FOR   THE    AN/ALR-67   RADAR   WARNING 

RECEIVER 

Litton    Applied    Technology   received   its   first 
contract  in  1975  under  a  cost  plus  fixed  fee  basis  from 
COMNAVAIRSYSCOM,   Washington,    D.C.,   to   develop   the 
AN/ALR-67   Radar   Warning   Receiver.     Due   to   budget 
constraints   at   the   Navy   level,  this  initial  concept 
exploration  contract  was  cancelled   three  times  between 
1975   and   1979.     During   the   interim  periods  Litton 
Applied  Technology  utilized  their  own  funds  to  continue 
the  development   with  a   skeleton  force   of  five  to  ten 
people.   Litton  estimates  they  spent   $400,000  of  their 
own   funds   during   1976   on  the  AN/ALR-67  development, 
with   the   hope   of   being   reimbursed.     The   initial 
contract    amount    was    for    $680,000.       Contract 
modifications   increased   this   amount    to   $1,000,000 
before   funds   were   temporarily   shut   down   in   1976. 
Subsequent   modifications,    eventually   totalling   100 
altogether,  increased   the  final  development  cost  total 
to  $6,530,000   under   the   contracted   amount.    Litton 
Applied    Technology     actually    spent    $10,541,541, 
incurring   a   $4,000,000   cost   overrun.     Ninety-four 
percent  of  the  development  effort  occurred  between  1975 
and  1978,  with  the   remainder  of   the  effort  continuing 
through  1980.    A   total  of  seven  prototype  models  were 
built  between  1975  and  1980. 

The  next  contract  covered  the  time  frame  from 
October  1982  to  December  1985.  This  contract  called 
for  limited  production  of  43  ALR-67  systems.  For  this 
production  start-up  endeavor  a  fixed  price  incentive 
type  contract  was  negotiated,  placing  more   of  the  cost 
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risk  on  Litton  Applied  Technology.  The  ceiling  price 
on  this  contract  wa  $46,600,000  for  the  43  systems. 

Since  1985,  a  full  scale  production  contract  has 
been  awarded  by  the  Navy  for  200  additional  systems  at 
a  total  price  of  $103,000,000.  This  contract  is  still 
ongoing . 

Table  4  presents  a  budget  and  actual  expenditure 
summary  for  contract  N00019-75-C-0390 .  The  actual 
development  costs  were  broken  into: 

Software  design  $  1,300,000 

Hardware  design  $  5,100,000 

Hardware  fabrication       $  2,200,000 

Data  $  1,100,000 

Test  requirements  $    800,000 

TOTAL  $10,500,000 

The  actual  costs  for  the  limited  production 
contract  were: 

Productionizing  $11,400,000 

Tooling  and  tests  $  5,400,000 

Data  $  3,000,000 

Hardware  $28,600,000 

TOTAL  $48,400,000 

G.   PREVIEW  OF  NEXT  CHAPTER 

Chapter  Five  analyzes  the  cost  data  through  a  brief 
variance  analysis  of  the  listed  cost  elements.  It  also 
includes  a  summary  of  the  findings  from  the  case  study 
interviews,  as  well  as  an  examination  of  possible  new 
directions  and  trends  for  cost  estimation  of  SOA 
proj  ects . 
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TABLE  4 

AN/ALR-6  7 

FUNDING  AND  EXPENDITURE  SUMMARY 

AUGUST  19  3  7   (MONTH  ENDING  3/2  3/37) 


EXPENDED 

:lin  § 

DESCRIPTION 

FUNDED 

TC 

)  DATE 

3e:ti:n  z 

DEVELOPMENT 

$6,530  ,000 

$10,541. 

541 

OPTION 

$1,255,566 

$ 

1,332. 

512 

3E  DTI 3N  F 

C002 

PGSE 

$3,514,134 

$ 

3,563. 

,040 

0003 

3 PARES /HARM 

$1,675,363 

$ 

1,711 

,861 

0007 

GSERDS 

$   332,054 

$ 

309 

,529 

0024 

ENG.  SUPPORT 

$1,749  ,008 

$ 

1,773 

,548 

00  2  5 

TEST  SUPPORT 

$4,252  ,944 

$ 

4,233 

,552 

0025 

ENG.  SERV/SYS 

S/W 

$3,527  ,127 

$ 

3,733 

,792 

0027 

FIELD  ENG.  SUPT 

$   200,000 

$ 

194 

,326 

0023 

TEST  SUPT  (M-l 

DEMO) 

$1,546,770 

$ 

1,304 

,389 

0023 

ENS.  SERV/SYS 

S/W 

$2  ,340,000 

9 

2  ,055 

,206 

0030 

7/ A- 13 A  INTGRTN 

$   146,607 

5 

148 

,464 

0  0  31 

F/A-13A  I/F 

TOTAL 

$   520,550 

$ 

471 

,000 

$27  ,640,773 

6  - 

51,427 

,760 
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V.   CASE  ANALYSIS  AND  FUTURE  TRENDS 

A.  INTRODUCTION 

This  chapter  provides  an  analysis  of  the  cost 
estimating  methodology  used  by  Litton  Applied 
Technology.  Emerging  trends  in  measuring  and 
controlling  SOA  costs  for  the  future  are  discussed. 
Specifically,  this  chapter  provides  the  following: 

1.  A  variance  analysis  of  the  AN/ALR-67  Radar 
Warning  Receiver  development  program. 

2.  An  overall  analysis  of  Litton  Applied 
Technology's  cost  estimating  system. 

3.  A  discussion  of  current  concerns  and  directions 
within  the  cost  estimating  field,  based  on  case 
study . 

4.  Thoughts  about  the  roles  cost  estimators  must 
assume  in  the  future. 

5.  Comparison  of  Litton's  cost  estimating  process  to 
the  current  direction  of  the  cost  estimating 
field. 

B.  VARIANCE  ANALYSIS  OF  THE  DEVELOPMENT  PROGRAM  OF  THE 
AN/ALR-67  RADAR  WARNING  RECEIVER 

The  majority  of  cost  overruns  for  the  current  life 
cycle  of  the  AN/ALR-67  Radar  Warning  Receiver  occurred 
in  the  conceptual  exploration  period  between  1975  and 
1980.  Variance  analysis  of  the  development  costs 
indicates  a  cost  overrun  of  $4,011,541,  or  61.4 
percent.  The  second  largest  contributor  to  the  total 
cost  variance  was  engineering  services  and  systems 
software  support,  which  overran  the  funded  amount  by 
5.8  percent,  or  $206,665.  Schedule  variances  could  not 
be  calculated  from  the  data  provided,  since  the 
category  for  funded  costs   was   not   further  subdivided 
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into  budgeted   cost  of  work  scheduled  and  budgeted  cost 
of  work  performed. 

From  discussions  with  company  officials  [Ref.  27], 
the  primary  reasons  for  the  cost  variances  in  the 
development  phase  were: 

1.  The  sporadic  nature  of  government  funding. 
During  the  years  1975-1979,  the  project  was 
stopped  three  times  by  the  Navy  due  to  funding 
constraints.   The  miscellaneous  work  stoppage  and 
startup  costs  associated  with  this  type  of 
uncertainty  were  not  completely  compensated  for 
by  the  Navy. 

2.  Poor  software  cost  estimating.   The  AN/ALR-67  was 
the  first  Litton  Applied  Technology  system  to 
incorporate  extensive  software  technology. 
Several  cost  analysts  stated  the  amount  of  labor 
hours  estimated  to  write  the  lines  of  code  for 
the  software  development  programs  were 
significantly  underestimated.   There  were  no 
analogous  programs  to  refer  to  for  cost  history 
comparisons.   Litton  Applied  Technology  does  not 
utilize  any  generic  cost  estimation  model  in 
software  or  hardware. 

3.  Inadequate  definition  of  the  work  breakdown 
structure.   During  the  years  1975-1980,  the  work 
breakdown  structure  was  developed  by  painstaking 
manual  methods ,  rather  than  use  of  the  current 
on-line  drafting  capability.   Litton  engineers 
found  it  difficult  during  the  initial  development 
phase  to  segregate  the  AN/ALR-67  prototype  models 
into  clearly  defined  lower  level  elements.   This 
lack  of  definition  led  to  problems  of 
underallocation  of  funds  as  actual  costs  began 
rolling  in. 
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4.  Inflation  and  its  effects  on  material  costs.   The 
years  1975-1980  were  periods  of  rapid  price 
increase  for  many  defense  electronics  materials. 
As  delays  due  to  work  stoppage  mounted  in  these 
initial  years,  original  material  estimates  became 
outdated.   As  a  result,  material  prices  were 
notably  higher  when  purchased  at  the  point  of 
usage . 

5.  Effect  of  experience.   The  various  work  stoppages 
during  the  development  phase  hindered  the 
learning  curve  progression  to  an  unmeasurable 
degree.   As  a  result,  more  rework  than  originally 
planned  occurred,  especially  in  software 
development . 

6.  Numerous  contract  modifications.   Litton  budget 
analysts  estimated  that  300  modifications  due  to 
engineering  change  proposals  were  added  to  the 
AN/ALR-67  development.   These  modifications 
accounted  for  the  funded  increase  to  $6,500,000 
from  the  original  contract  amount  of  $680,000. 
Most  modifications  involved  increased  integration 
of  circuits  to  provide  added  performance  within 
the  same  size  constraints. 

7.  Schedule  pressure.   During  1978,  considerable 
pressure  was  exerted  by  the  Navy  to  accelerate 
fabrication  of  the  engineering  development 
models.   This  pressure  led  to  an  unplanned 
increase  in  the  number  of  workers  and  an  increase 
in  rework,  both  of  which  contributed  to  cost 
increases . 

8.  Lack  of  formal  controls.   Litton  Applied 
Technology  did  not  have  a  well-documented  control 
system  of  checklists  and  unplanned  audits  in  the 
period  1975-1980.   Internal  investigations  of 
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variance  were  not  available  for  review  for 
contracts  during  this  period.   This  era  also 
predated  the  DOD  procurement  reform  measures  of 
the  mid-1980's,  which  required  more  stringent 
control  systems  by  government  contractors. 

C.   OVERALL  ANALYSIS   OF  LITTON  APPLIED  TECHNOLOGY'S 

COST  ESTIMATING  SYSTEM 

This  analysis  of  the  cost  estimating  system  for  the 
AN/ALR-67  Radar  Warning  Receiver  and  Litton  Applied 
Technology  is  based  on  the  six  managerial  subsystem 
characteristics  of  the  Katz  and  Rosenzweig  Model,  which 
are  environmental,  technical,  goals  and  values, 
psychological,  managerial,  and  structural.   [Ref.  27] 

1 .   Environmental 

The  environment  which  influences  Litton  Applied 
Technology  is  much  different  in  1987  than  it  was  during 
1980,  the  end  of  the  development  period.  In  1980, 
Applied  Technology  was  still  a  separate  company  from 
Litton,  so  it  could  not  depend  on  large  corporate 
resources  for  assistance.  Applied  Technology 
encountered  erratic  government  funding  for  its  first 
major  weapons  system  to  use  software  extensively. 
Also,  competition  from  other  defense  electronics 
companies  was  not  as  intense  in  1980  as  in  1987.  For 
example,  the  concepts  of  contractor  teaming,  dual 
sourcing,  and  leader-follower  for  development  projects 
were  not  introduced  to  the  defense  industry  until 
several  years  after  the  conceptual  exploration  phase 
for  the  AN/ALR-67  was  completed.  These  external 
environmental  factors  which  differentiate  the  period 
1975-1980  from  1987  allow  insight  into  how  Applied 
Technology  has  evolved  to  meet  the  current  defense 
climate  via   structural  and  technical  changes.   Defense 
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contractors,  in  1987,   face   more   intense   scrutiny  on 
cost   performance   than   they   encountered   in  1980  for 
development  contracts. 
2 .   Technical 

Litton  Applied  Technology  has  greatly  improved 
its  cost  estimating  capabilities.  They  currently  use  a 
new  IBM  mainframe  computer  system  to  keep  track  of 
price  histories,  work  breakdown  structures,  and  vendor 
quotes.  However,  they  have  made  a  conscious  decision 
not  to  utilize  generic  parametric  cost  estimating 
models  which  could  be  adapted  to  the  mainframe  system. 
Litton  cost  analysts  believe  the  physical  dimensions  of 
their  radar  warning  receivers  are  too  small  for 
accurate  application  of  the  RCA  Price  cost  estimating 
model,  for  example.  As  a  result,  all  cost  improvements 
are  geared  toward  upgrading  their  bottoms-up  cost 
estimating  capability.  Their  emphasis  is  on  cost 
performance  trend  analysis  for  their  own  products. 

Litton  Applied  Technology's  cost  estimators 
understand  the  difficulties  of  accurate  estimation  for 
the  new  integrated  circuits  of  the  future.  The  SOA  for 
Litton's  radar  warning  receivers  will  extend  into 
increasing  miniaturization  of  components,  which  will  be 
more  reliable  and  capable  than  their  predecessors. 
Litton  estimators  currently  expect  vendor-purchased 
electronic  components  to  decrease  in  price  in  the  next 
few  years  due  to  better  integrated  circuit  technology 
and  yield  improvement.  Litton  will  continue  to  rely  on 
its  mainframe  cost  collection  system  and  group  judgment 
techniques  to  estimate  costs  for  SOA  extensions,  with 
no  foreseeable  plans  to  incorporate  regression  analysis 
or  cost  prediction  models.  Finally,  due  to  its 
mainframe  capabilities,  Litton  rarely  uses  personal 
computers  in  its  cost  estimation  process. 
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3 .  Structure 

Structurally,  Litton  Applied  Technology 
utilizes  a  matrix  organization  concept  for  its  cost 
estimation  process.  With  this  system  they  draft 
members  from  functional  departments  to  participate  in 
the  cost  proposal  process,  terminating  the  team  upon 
completion  of  the  negotiated  contract. 

Litton  Applied  Technology  has  the  functional 
department  heads  review  and  approve  the  proposal  team's 
recommendations.  This  current  system  does  not  allow 
any  one  individual  the  opportunity  to  wield  an 
overwhelming  influence  on  cost  estimates.  The 
situation  differed  slightly  from  1975-1980,  when  the 
AN/ALR-67  program  management  team  remained  intact  and 
exerted  considerable  independence. 

4 .  Psychosocial 

At  the  psychosocial  level,  Litton  Applied 
Technology  has  made  tremendous  strides  for  its 
employees.  At  its  south  San  Jose  location,  a  new 
building  with  plush  offices,  a  complete  cafeteria,  and 
recreational  facilities  have  been  added  to  improve 
company  morale.  Litton  requires  all  employees, 
including  executives,  to  refer  to  each  other  by  first 
names  only.  Litton's  executives  also  meet  annually  at 
special  retreats  to  discuss  the  current  state  of  the 
company  and  its  future  strategy. 

Most  of  Litton's  cost  estimators  are  relatively 
young,  with  an  accounting  background  from  college. 
Litton  trains  these  estimators  themselves,  rather  than 
relying  on  outside  cost  estimating  seminars.  They  do 
not  participate  in  the  Space  Systems  Cost  Analysis 
Group  or  belong  to  the  International  Society  of 
Parametric  Analysts.  Litton  has  recently  published  a 
comprehensive  "Pricing   and  Estimation   Manual"  for  its 
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cost   analysts.     However,    in    1980,    few   written 
guidelines  were  available  for  use  by  the  cost  analysts. 

5 .  Managerial 

The  managerial  process  at  Litton  Applied 
Technology  is  heavily  weighted  toward  planning  and 
organization.  Almost  every  area  of  the  building 
carries  its  own  blackboard  and  space  for  meeting  rooms. 
According  to  the  Pricing  and  Estimation  Manual,  most 
meetings  have  set  time  limits  of  five  to  ten  minutes 
per  topic.  Litton  exhibits  a  strong  vertical  decision- 
making structure,  since  functional  department  levels 
usually  do  not  deal  with  other  functional  lower  levels 
unless  prior  liaison  has  been  arranged  by  the 
department  heads. 

6 .  Goals  and  Values 

Litton  Applied  Technology's  goals  and  values 
emphasize  quality  workmanship  and  reliability.  Their 
marketing  personnel's  primary  emphasis  is  on  the 
excellent  performance  and  versatility  of  the  radar 
warning  receivers  in  combat  situations.  At  proposal 
team  meetings,  the  actual  cost  estimates  are  usually 
not  the  primary  issue.  Proposal  meeting  agendas 
concentrate  on  analysis  of  the  competition  and 
packaging  and  presentation  of  the  proposal  itself. 

7 .  Control  Mechanisms 

The  AN/ALR-67's  costs  are  currently  controlled 
primarily  through  vendor  control  and  monitoring  of 
labor  hours.  Litton  Applied  Technology  has  established 
a  vendor  qualification  program  to  meet  its  raw  material 
needs,  in  order  to  minimize  material  defects.  Vendor 
costs  are  controlled  by  trend  comparison  with  prior 
related  efforts  on  the  mainframe  history  files.  Litton 
Applied   Technology's   internal   auditors   monitor,  via 
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surprise  visits,   the  recording   of  labor  hours  by  each 
cost  element. 

In  this  researcher's  opinion,  Litton  Applied 
Technology  could  more  effectively  utilize  postaudits  as 
a  control  mechanism.  Their  cost  analysts  receive 
little  feedback  on  proposal  cost  estimates  which  are 
rejected  in  favor  of  another  contractor.  Postaudit 
conferences  with  cost  analysis  can  provide  an  effective 
"lessons  learned"  benefit  for  the  future. 

D.   CURRENT  CONCERNS  AND  TRENDS  IN  COST  ESTIMATION 

The  current  focus  of  practitioners  of  SOA  cost 
estimating  is  on  increasing  the  range  and  depth  of  the 
data  base.  During  an  interview  with  Dr.  E.N.  Dodson  on 
16  October  1987,  he  indicated  a  primary  problem  in  cost 
estimating  is  the  lack  of  good  historical  data  to 
substantiate  cost  prediction  models.  Dodson  indicated 
most  data  for  cost  models  comes  from  after-the-fact 
analysis  of  costs  at  the  production  level.  Dodson 
feels  more  rigorous  analysis  of  costs  at  the  design 
stage  is  critical  for  accounting  for  technological 
change.  To  properly  understand  the  cost  impact  of  new 
technology,  Dodson  stated  cost  estimators  must  become 
more  knowledgeable  in  engineering.  Engineering 
backgrounds  would  enable  estimators  to  better  locate 
the  cost  drivers  which  are  influenced  by  technology 
parameters . 

Another  concern  of  modern  cost  estimators, 
according  to  Dr.  Dodson,  is  the  inability  to  influence 
potential  costs  during  the  initial  phases  of  the  design 
process.  Dodson  feels  cost  estimators  should  develop 
the  capability  to  review  performance  parameters  in  the 
design  process,  and  subsequently  advise  design 
engineers  as   to  the   legitimacy  of  the  specifications. 
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For  example,  cost  analysts  should  be  capable  of  asking 
If  less  rigorous  specifications  could  be  substituted 
for  more  rigorous  ones. 

One  shortcoming  of  many  cost  estimators,  according 
to  Dr.  Dodson,  is  their  failure  to  recognize  old 
technology  under  the  guise  of  new  technology.  Once 
estimators  increase  their  technical  knowledge,  this 
mistake  will  be  less  likely  to  occur.  Also,  better 
subdivision  of  the  work  breakdown  structure  into 
definable  cost  elements  should  sort  out  old  and  new 
technology. 

Dodson  feels  his  theories  on  the  use  of  surface 
fitting  techniques  and  regression  analysis  to  estimate 
SOA  extension  costs  are  too  time-consuming  and 
expensive.  Most  corporations  involved  in  government 
cost  proposals  could  not  efficiently  utilize  these 
techniques.  As  Dodson  indicates,  DOD  agencies  prefer 
to  audit  detailed  systems  engineered  estimates  which 
substantiate  every  cost  element.  DOD  agencies  do  not 
require  parametric  estimates  of  new  developments,  so 
cost  estimators  currently  use  cost  estimation  models 
only  as  a  checking  mechanism. 

For  SOA  extension  measurement,  Dodson  states  his 
current  focus  is  on  transforming  performance  parameters 
to  design  parameters,  and  ultimately  to  cost.  He 
emphasizes  efficiency  parameters  should  be  utilized  in 
selecting  key  technological  variables.  For  example,  he 
suggests  receiver  sensitivity,  receiver  bandwidth,  and 
receiver  frequency  might  be  good  indicators  of  SOA 
advance  for  radar  systems. 

Finally,  Dr.  Dodson  states  risk  analysis  of  cost 
estimates  is  another  area  which  requires  further  study. 
This   area   should   quantify    the   probabilities   that 
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underruns   or    overruns   will    occur   in   development 
proj  ects . 

Stan  Swales,  the  leading  cost  estimator  for  GTE 
Government  Products  and  a  member  of  the  Space  Systems 
Cost  Analysis  Group,  also  shared  his  opinion  on  the 
current  trends  of  the  cost  estimating  field.  [Ref.  28] 
Mr.  Swales  recommends  the  use  of  Expert  Systems,  a 
branch  of  artificial  intelligence  that  is  concerned 
with  emulating  the  problem  solving  processes  of  human 
experts.  On  28  July  1987  he  stated  "Expert  Systems  is 
the  wave  of  the  future."  He  feels  Expert  Systems 
technology  optimizes  the  exchange  of  information  and 
improves  the  evaluation  of  data.  For  Mr.  Swales' 
system,  cost  engineers  interview  experts  and  program 
their  answers  into  the  knowledge  base  of  the  Expert 
System.  The  knowledge  base  should  eventually  consist 
of  rules,  cost  estimating  relationships,  and  a  numeric 
data  base.  The  next  phase  of  Swales'  Expert  System 
concerns  the  inference  engine,  which  contains  the 
control  strategies  and  control  structures  for  the 
model.  The  cost  predictor  portion  of  the  model 
develops  cost  estimates  for  programs  by  combining  the 
control  given  by  the  inference  engine  with  the  user's 
input  data  and  the  knowledge  base.  Although  Swales 
utilizes  this  model  to  predict  costs  of  programs  in  the 
conceptual  exploration  stage,  it  has  not  been 
completely  implemented  into  GTE ' s  formal  cost 
estimating  system.  The  current  application  of  his 
Expert  System  does  not  specifically  quantify  SOA 
extension  costs. 

Stan  Swales  states  Expert  Systems  solve  cost 
estimating  problems  by  heuristic,  or  rules  of  thumb 
methods.  He  believes  Expert  Systems,  with  their  non- 
algorithmic    solutions,    can    more    closely    mimic 
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judgmental  cost  estimating  process  than  conventional 
methods  can.  In  his  presentation  on  Expert  Systems  to 
the  Space  Systems  Cost  Analysis  Group  on  24  September 
1986,  he  drew  three  conclusions: 

1.  Expert  Systems  rapidly  adapt  to  any  type  program, 
with  benefits  increasing  exponentially  with 
program  complexity. 

2.  Expert  Systems  cost  models  designed  for  desk-top 
computers  reduce  proposal  times  and  improve 
program  cost  control . 

3.  Expert  Systems  techniques  reduce  potential  for 
program  cost  overruns. 

E.   EMERGING  ROLES   COST  ESTIMATORS  MUST  ASSUME  IN  THE 

FUTURE 

This  researcher  discovered  through  personal 
interviews  with  Dr.  E.N.  Dodson  and  Stan  Swales  that 
most  leading  cost  estimators  have  also  attained  a  broad 
engineering  background.  During  a  presentation  to  the 
International  Society  of  Parametric  Analysts  during 
March  1987,  Mr.  Edward  Laughlin  discussed  the  role  cost 
estimators  must  assume  to  properly  assess  future 
technologies.  [Ref.  29] 

Laughlin  stated   cost  estimators  must  improve  their 

understanding   of   the    total    implications    of   new 

technology.    He  believes   analysts  must  develop  closer 

coordination  with   the  engineering   community  to  better 

understand   the    technical   implications   of   resource 

management   decisions.     They    must    be    fluent   in 

acquisition,   budgetary,   and  technical  languages  since 

the  cost  analyst  will   become   an   integrator   of  staff 

information   in   the   future.     For   example,  Laughlin 

writes : 

We  can't   afford   to   look   blank   when   people  start 
talking   about   flexible  manufacturing,  fiber  optics, 
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robotics,  and  composite  materials  because  we're  going 
to  be  tasked  to  recommend  important  and  costly 
decisions  that  can't  be  made  with  yesterday's 
knowledge.  [Ref.  30] 

Laughlin    believes    cost    estimators   must   have 

diversified  experience  and  a   broad  based   education  to 

understand   the   cost   impact   of   new   technology.   He 

recommends  that  data  bases   concentrate  on  accumulating 

information  on   technological  cost  drivers,  rather  than 

massive  storage  of  cost  histories.    He  also  recommends 

that  cost  estimators  continually  update  their  education 

and  training  on  future   hardware   and   software  trends, 

such   as   computer   aided   design   (CAD)  and  artificial 

intelligence.   Finally,  Laughlin  recommends  engineering 

specialists  become   permanent  members   of  cost  analysis 

divisions   to   increase   the    interdependency   between 

engineers  and  cost  estimators. 

F.   COMPARISON  OF  LITTON'S  COST  ESTIMATING  PROGRAM  TO 
CURRENT  DIRECTION  OF  THE  ESTIMATING  FIELD 
Litton  Applied   Technology's  cost  estimating  system 
has   evolved    into   a    bottoms-up   system   engineered 
procedure  which  makes  extensive  use  of  a  deep  data  base 
of  prior  related  costs.    Their   data  base   consists  of 
price   histories   for   parts   which   date   back  fifteen 
years.   Extensive  labor  rate  data  for  carefully  defined 
job  descriptions  are  also  entered  into  the  data  base. 
Their  ability   to  adapt   to  the  future  direction  of  the 
cost  estimating  field,  and  their  capability  to  estimate 
the  cost  of  SOA  advances,  are  assessed  in  the  following 
paragraphs . 

1 .   SOA  Advance  Measurements 

Litton  does  not  specifically  measure  the  cost 
of  the  SOA  extension.  Dodson's  use  of  ellipsoid 
surfaces  to  define  the  SOA  performance  parameters  has 
not  been  addressed  by  Litton  Applied  Technology.   Other 
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SOA  measurement  techniques,  such  as  Alexander's 
regression  analysis  of  primary  parameters,  are  also  not 
utilized.  Even  for  conceptual  exploration  phase 
projects,  Litton  relies  on  consistent  application  of 
its  bottoms-up  system.  The  proposal  analyst  and  the 
functional  engineers  use  analogy  methods  when  exact 
data  from  the  WBS  elements  is  not  available.  Litton 
cost  estimators  do  not  use  generic  parametric  cost 
models  such  as  PRICE  or  FAST. 

This  researcher  feels  Litton's  expensive 
commitment  to  their  bottoms-up  method  for  SOA  extension 
projects  does  not  provide  their  analysts  sufficient 
flexibility  for  "what-if"  analysis.  Their  exploration 
of  different  alternatives  is  limited,  since  detailed 
documentation  efforts  at  the  WBS  lower  levels  are 
mandated  for  each  acquisition  phase.  Their  exclusive 
use  of  the  bottoms-up  method  does  not  provide 
information  on  the  expense  involved  in  increasing  SOA 
parameters,  such  as  receiver  sensitivity,  in  a  timely 
manner . 

2 .   Expert  Systems 

Artificial  intelligence  applications  to  cost 
estimating  will  become  more  prevalent  in  the  future. 
Litton  Applied  Technology,  with  its  extensive  cos  data 
base,  is  missing  an  opportunity  to  improve  its 
estimating  capability  by  not  investigating  the  benefits 
of  this  field.  They  already  possess  extensive  computer 
capabilities  which,  coupled  with  their  knowledge  of 
radar  warning  receivers,  makes  them  ideal  candidates 
for  expert  systems. 

Some  of  the  opportunities  the  use  of  expert 
systems  would  present  to  Litton  are  listed. 
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a.  Learning  Tool 

Expert  systems  give  users  the  capability  to 
obtain  the  experts'  consensus  Judgment  on  what  costs 
should  be,  based  on  the  users'  answers  to  system 
queries.  Litton's  cost  analysts  do  not  have 
postgraduate  education  or  extensive  engineering 
backgrounds,  but  implementation  of  an  expert  system 
would  rapidly  increase  their  capabilities  to  mimic  an 
expert's  answer. 

b.  Warehouse  for  Cost  Estimating  Knowledge 
The   expert   system's   knowledge   data  base 

would  incorporate  the  expertise  of  Litton's  most 
capable  professionals.  Their  proposal  analysis  teams 
would  gain  consistency  through  use  of  the  expert 
system,  since  recommendations  by  the  system  would  be 
based  on  the  same  heuristic  rules  input  to  the  system. 

c.  Pre-planning  Tool 

Expert  systems,  normally  designed  for 
personal  computers,  provide  the  rapid  flexibility  of 
what-if  analysis  missing  in  Litton  Applied  Technology's 
current  system. 

d.  Research  Tool 

Litton's  cost  estimators  could  perform 
research  on  new  projects  by  studying  the  implications 
of  different  parameters  in  a  quick,  iterative  fashion. 

Expert  systems  are  expensive,  and  most  are 
still  in  formative  stages.  This  researcher  recommends 
Litton  track  the  progress  of  artificial  intelligence  in 
the  cost  estimating  field.  Cost  benefit  analysis 
should  be  performed  to  see  if  their  data  based 
management  system  would  be  significantly  improved. 
3 .   Influence  of  Costs  in  the  Design  Stage 

Cost   estimators   at   Litton   do  not  yet  impact 
planning  of  performance  parameters  at  the  design  stage. 
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In  the  proposal  cycle  the  cost  analyst  collects  Inputs 
from  the  WBS  elements,  after  the  specifications  are 
already  decided.  However,  with  the  rapid  pace  of 
technology,  Litton  will  gain  more  effective  control  of 
costs  through  more  rigorous  analysis  of  costs  earlier 
in  the  design  stage. 

4 .  Understanding  of  Total  Implications  of  New 
Technology 

Most  Litton  cost  analysts  have  accounting 
backgrounds.  They  do  not  regularly  attend  professional 
cost  estimating  seminars.  For  Litton  to  improve  its 
ability  to  estimate  SOA  extension  costs,  the  estimators 
must  understand  how  new  technology  will  influence 
costs.  The  cost  analysts  should  receive  training  on 
fields  such  as  artificial  intelligence  and  computer 
aided  design.  This  researcher  recommends  high 
technology  firms  like  Litton  should  consider  inclusion 
of  engineers  knowledgeable  In  future  hardware  trends  in 
the  cost  estimating  division. 

5 .  Isolation  of  Cost  Drivers 

Litton's  ability  to  estimate  SOA  extension 
costs  and  control  development  costs  would  be  enhanced 
by  a  thorough  analysis  of  which  design  and  performance 
parameters  generate  the  most  significant  influence  on 
cost.  Isolation  of  the  key  cost  drivers  for  radar 
warning  receivers  would  enable  planners  to  concentrate 
on  cost  reductions  in  these  areas. 

F.   SUMMARY 

This  chapter's  primary  goal  has  been  to  analyze  key 
features  of  Litton  Applied  Technology  through  specific 
study  of  the  AN/ALR-67's  development  program. 
Preceding  sections  provided  insight  into  the  direction 
of  the   cost  estimating   field,  and   the  qualities  cost 
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analysts   must   develop   in  the  future,  before  reliable 
cost  estimates  for  SOA  extensions  can  be  made. 

The  next  chapter   provides   the   final   summary  and 
recommendations  for  future  study. 
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VI .   SUMMARY  AND  CONCLUSIONS 

A.  INTRODUCTION 

This  chapter  presents  the  summary  and  conclusions 
for  this  thesis.  The  primary  focus  of  the  research 
has  been  to  determine  how  defense  industry  cost 
estimators  measure  the  costs  of  SOA  extensions  for  new 
weapons  systems.  The  researcher  utilized  a  case  study 
of  Litton  Applied  Technology's  development  of  the 
AN/ALR-67  Radar  Warning  Receiver  to  present  an  actual 
example . 

The  literature  review  section  highlighted  the  work 
of  Dr.  E.N.  Dodson  of  General  Research  Corporation,  a 
pioneer  in  the  field  of  SOA  measurement.  Cost 
prediction  models  by  selected  authors  were  also 
examined  to  demonstrate  how  cost  estimation  theory  is 
transformed  to  actual  data.  The  last  chapter 
highlighted  current  concerns  and  future  trends  in  the 
cost  estimating  field. 

B.  LITERATURE  REVIEW 

The  literature  review  section  began  with  Ostwald's 
definition  of  cost  estimating  relationships  as 
functional  models  that  mathematically  describe  the 
costs  of  components  as  functions  of  one  or  more 
independent  variables.  Ostwald  also  described  the 
roles  cost  estimators  play  in  technological  forecasting 
and  development  of  cost  indices. 

Dr.  E.N.  Dodson  has  established  himself  as  one  of 
the  foremost  practitioners  of  quantitative  measurement 
of  SOA  advances.  He  described  the  SOA  for  a  particular 
system  as  an  n-dimensional  function  at  a  particular 
point  in  time,  with  n  representing  the  number  of  SOA 
design   characteristics.       He    indicated   geometric 
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surfaces  could  illustrate  the  tradeoff  between  design 
characteristics  for  SOA  advances.  Dodson  specifically 
developed  SOA  equations  for  solid  propellant  missiles. 
He  also  constructed  SOA  indices  by  geometric  measures 
of  data  characteristics  on  an  ellipsoid  surface. 

Alexander  and  Nelson  of  Rand  Corporation  measured 
technological  change  through  multiple  regression 
analysis.  Their  studies  indicated  performance,  rather 
than  technical ,  parameters  were  better  descriptors  of 
SOA  advance. 

Hovanssian  recommended  electronic  systems  include 
customer  acceptance  parameters  at  the  development 
stage,  such  as  operator  approval,  life  cycle  cost,  and 
amount  of  maintenance  required  per  operating  hour. 
Cooley  developed  the  TRAM  model,  which  could  analyze 
the  effects  of  budget  reductions  on  SOA  development 
pro j ects . 

Other  studies  of  technological  advance  were 
conducted  by  Gordon  and  Munson ,  who  used  experts  to 
choose  the  proper  parameters  for  their  SOA  extension 
equations.  Finally,  Marino  further  refined  the  role  of 
surface  analysis  in  SOA  measurements. 

C.   COST  MODELS 

Four  different  cost  estimating  models  have  been 
examined  in  earlier  chapters.  The  two  primary 
ingredients  in  their  development  were: 

1.  Derivation  of  the  work  breakdown  structure. 

2.  Development  of  cost  estimating  relationships  for 
each  cost  element. 

The  cost  estimating  relationships  at  the  lowest 
level  of  the  WBS  were  derived  by  a  variety  of  methods 
for  these  models,  from  parametric  analysis  to  analogy 
comparisons.   The  cost  models  all   provided  a  hierarchy 
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of  estimating  systems,  depending  on  the  data  available 
to  the  analyst.  In  Dodson's  models,  he  recommended 
regression  equations  should  only  be  used  as  a  last 
resort . 

The  FAST  model  utilizes  technological 
characteristics  such  as  energy  outputs  for  its 
foundation.  Among  the  products  of  the  FAST  model  are 
projections  of  funding  requirements  for  each  stage  of 
the  development  cycle. 

D.   LITTON' S  COST  ESTIMATION  SYSTEM 

Litton  Applied  Technology  utilizes  a  detailed 
bottoms-up  cost  estimation  approach  for  all  electronics 
systems  projects,  including  those  at  the  early 
development  stage.  They  do  not  use  parametric  cost 
estimation  models;  they  rely  on  an  elaborate  component 
cost  history  data  dating  back  to  1975stored  on  their 
IBM  Mainframe  computer.  Their  cost  estimation  process 
involves  eliciting  judgments  from  many  personnel  from 
different  functions  organized  in  a  matrix  team 
structure.  The  final  cost  estimate  decision  is  arrived 
at  only  after  a  series  of  judgmental  steps  at  different 
committee  meetings. 

Litton  Applied  Technology  experienced  a  cost 
overrun  of  61  percent  for  the  development  phase  of  the 
AN/ALR-67  Radar  Warning  Receiver.  Some  of  the  reasons 
explaining  this  cost  growth  were  Litton  Applied 
Technology's  lack  of  familiarity  with  software  cost 
estimation,  numerous  additions  of  modifications, 
intermittent  work  stoppages,  and  poor  definition  of 
manually  drafted  work  breakdown  structures.  They  did 
not  use  any  specific  techniques  or  parametric  cost 
models  to  estimate  the  actual  cost  of  the  SOA 
extension . 
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E.  CURRENT  DIRECTIONS 

Multiple  regression  analysis,  trade  off  surface 
analysis,  and  factor  analysis  have  been  used  by  various 
practitioners  to  measure  and  estimate  the  actual  costs 
of  SOA  extensions.  Research  has  now  focused  on 
analysis  of  the  efficiency  characteristics  of  the 
primary  parameters  which  most  completely  describe  the 
technological  advancement.  Interviews  with  Dr.  Dodson 
suggest  the  results  of  these  SOA  extension  studies  have 
not  been  integrated  into  the  current  cost  estimation 
models  . 

Other  cost  estimators,  like  Stan  Swales,  predict 
artificial  intelligence  will  be  used  more  extensively 
in  cost  estimation  models  in  the  future.  However, 
there  are  no  indications  Expert  Systems  methodology  has 
specifically  addressed  the  issue  of  SOA  extension 
measurement  in  actual  weapons  systems  applications. 

F.  FUTURE  ROLES 

The  cost  estimator  of  the  future  must  assume  the 
role  of  an  integrator  between  engineering  and  cost 
analysis.  With  many  electronic  hardware  costs  actually 
decreasing  exponentially  due  to  technological 
innovation,  cost  estimators  must  continually  update 
their  knowledge  on  the  latest  technological 
innovations.  Data  bases  should  contain  information  on 
the  cost,  performance,  and  physical  characteristics  of 
the  primary  cost  drivers  which  influence  technological 
advancement . 

DOD  project  offices  can  reduce  overruns  in  SOA 
development  programs  by  decreasing  the  amount  of 
administrative  and  specification  changes  during  the 
conceptual  exploration   stage.    However,  to  facilitate 
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this,  government  cost  estimators  must  become  involved 
early  in  the  design  process,  and  advise  engineers  as  to 
the  cost  tradeoffs  of  specification  changes. 

G.   CONCLUSIONS 

The  following  conclusions  can   be   drawn   from  this 
study : 

1.  The  systems   engineered  cost  estimating  approach 
is  relied  upon  by  Litton  Applied  Technology  for 
its  cost  estimation  methodology. 

2.  Various  methods,  such  as  Dr.  Dodson's  regression 
analysis  and  surface  fitting  techniques,  have 
succeeded  in  measuring  the  cost   of   SOA 
extensions.   These  methods  have  not  been  utilized 
in  actual   applications  by  industry  or  DOD  in 
their  cost  estimating  models. 

3.  The  fundamental  ingredient  to  parametric  analysis 
of  technological  advancements  is  specification  of 
the  relationship  between  cost,  performance,  and 
physical  characteristics  of  the  primary  cost 
drivers . 

4.  The  development  program  cost  estimate  requires  a 
detailed  work  breakdown  structure  in  a  bottoms-up 
method  to  prevent  cost  omissions.   Errors  in  this 
area  are  a  frequent  cause  of  cost  overruns. 

5.  Cost  estimators  must  possess  technical  knowledge 
of  the  task  being  estimated  if  they  are  going  to 
competently  measure  the  costs  of  technological 
advancement . 

6.  The  cost   estimating  field   is  dynamically 
increasing   its   knowledge   in   the   area  of 
parametric  desk-top  models  and  expert  systems. 
These  systems  are  useful   for   rapid   analysis  of 
different  possibilities. 
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H.   RECOMMENDATIONS  FOR  FUTURE  STUDY 

Future  thesis  efforts  are  needed  to  track  the 
progress  of  the  cost  estimating  profession  in  its 
endeavor  to  quantify  the  costs  of  SOA  extensions. 
Also,  future  studies  should  review  the  feasibility  of  a 
large,  universal  data  base  for  all  electronics  systems, 
from  which  defense  contractors  and  DOD  cost  estimators 
could  extract  information.  Pooled  efforts  might 
accelerate  the  progress  in  transforming  technological 
advancement  theory  to  actual  practice. 
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